-- Logs begin at Tue 2022-06-07 21:16:28 UTC, end at Tue 2022-06-07 22:23:46 UTC. -- Jun 07 21:48:26.271162 dv-cinder-835525-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 07 21:48:26.279144 dv-cinder-835525-4-iscsi devstack@keystone.service[80433]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 07 21:48:26.281175 dv-cinder-835525-4-iscsi devstack@keystone.service[80433]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Jun 7 21:48:26 2022] *** Jun 07 21:48:26.281175 dv-cinder-835525-4-iscsi devstack@keystone.service[80433]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 07 21:48:26.281175 dv-cinder-835525-4-iscsi devstack@keystone.service[80433]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 07 21:48:26.281175 dv-cinder-835525-4-iscsi devstack@keystone.service[80433]: nodename: dv-cinder-835525-4-iscsi Jun 07 21:48:26.281175 dv-cinder-835525-4-iscsi devstack@keystone.service[80433]: machine: x86_64 Jun 07 21:48:26.281175 dv-cinder-835525-4-iscsi devstack@keystone.service[80433]: clock source: unix Jun 07 21:48:26.281175 dv-cinder-835525-4-iscsi devstack@keystone.service[80433]: pcre jit disabled Jun 07 21:48:26.281628 dv-cinder-835525-4-iscsi devstack@keystone.service[80433]: detected number of CPU cores: 4 Jun 07 21:48:26.281628 dv-cinder-835525-4-iscsi devstack@keystone.service[80433]: current working directory: / Jun 07 21:48:26.281628 dv-cinder-835525-4-iscsi devstack@keystone.service[80433]: detected binary path: /usr/bin/uwsgi-core Jun 07 21:48:26.281628 dv-cinder-835525-4-iscsi devstack@keystone.service[80433]: your processes number limit is 47680 Jun 07 21:48:26.281628 dv-cinder-835525-4-iscsi devstack@keystone.service[80433]: your memory page size is 4096 bytes Jun 07 21:48:26.281628 dv-cinder-835525-4-iscsi devstack@keystone.service[80433]: detected max file descriptor number: 2048 Jun 07 21:48:26.281628 dv-cinder-835525-4-iscsi devstack@keystone.service[80433]: lock engine: pthread robust mutexes Jun 07 21:48:26.281628 dv-cinder-835525-4-iscsi devstack@keystone.service[80433]: thunder lock: enabled Jun 07 21:48:26.281628 dv-cinder-835525-4-iscsi devstack@keystone.service[80433]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 07 21:48:26.281628 dv-cinder-835525-4-iscsi devstack@keystone.service[80433]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 07 21:48:26.327060 dv-cinder-835525-4-iscsi devstack@keystone.service[80433]: Python main interpreter initialized at 0x5621ec1559c0 Jun 07 21:48:26.327060 dv-cinder-835525-4-iscsi devstack@keystone.service[80433]: python threads support enabled Jun 07 21:48:26.327060 dv-cinder-835525-4-iscsi devstack@keystone.service[80433]: your server socket listen backlog is limited to 100 connections Jun 07 21:48:26.327060 dv-cinder-835525-4-iscsi devstack@keystone.service[80433]: your mercy for graceful operations on workers is 90 seconds Jun 07 21:48:26.327458 dv-cinder-835525-4-iscsi devstack@keystone.service[80433]: mapped 403077 bytes (393 KB) for 2 cores Jun 07 21:48:26.327557 dv-cinder-835525-4-iscsi devstack@keystone.service[80433]: *** Operational MODE: preforking *** Jun 07 21:48:26.327835 dv-cinder-835525-4-iscsi devstack@keystone.service[80433]: *** uWSGI is running in multiple interpreter mode *** Jun 07 21:48:26.327835 dv-cinder-835525-4-iscsi devstack@keystone.service[80433]: spawned uWSGI master process (pid: 80433) Jun 07 21:48:26.328044 dv-cinder-835525-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 07 21:48:26.329130 dv-cinder-835525-4-iscsi devstack@keystone.service[80433]: spawned uWSGI worker 1 (pid: 80445, cores: 1) Jun 07 21:48:26.329130 dv-cinder-835525-4-iscsi devstack@keystone.service[80433]: spawned uWSGI worker 2 (pid: 80446, cores: 1) Jun 07 21:48:26.329130 dv-cinder-835525-4-iscsi devstack@keystone.service[80433]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 07 21:48:32.049426 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 07 21:48:32.049426 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: from cryptography.utils import int_from_bytes Jun 07 21:48:32.050183 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 07 21:48:32.050183 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: from cryptography.utils import int_from_bytes Jun 07 21:48:32.050594 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 07 21:48:32.050594 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: from cryptography.utils import int_from_bytes Jun 07 21:48:32.050713 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 07 21:48:32.050713 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: from cryptography.utils import int_from_bytes Jun 07 21:48:32.100297 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 07 21:48:32.100585 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 07 21:48:32.100670 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 07 21:48:32.100761 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 07 21:48:32.100845 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] command line args: [] {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 07 21:48:32.100964 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] command line args: [] {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 07 21:48:32.101096 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 07 21:48:32.101212 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ================================================================================ {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 07 21:48:32.101309 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 07 21:48:32.101388 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ================================================================================ {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 07 21:48:32.101467 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] admin_token = **** {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.101630 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] admin_token = **** {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.101710 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] config_dir = [] {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.101875 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] config_dir = [] {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.101954 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.102055 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] config_source = [] {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.102154 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.102234 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] config_source = [] {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.102304 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] debug = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.102401 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] debug = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.102503 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.102682 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.102776 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.102844 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.103035 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.103129 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.103196 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.103255 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.103452 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.103536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.103686 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] list_limit = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.103784 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] list_limit = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.103851 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] log_config_append = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.103912 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] log_config_append = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.104062 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.104141 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.104206 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] log_dir = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.104271 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] log_dir = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.104335 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] log_file = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.104394 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] log_file = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.104563 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.104641 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.104806 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.104904 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.105051 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.105126 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.105290 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.105376 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.105452 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.105520 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.105701 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.105822 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.105913 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.105995 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.106099 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.106200 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.106331 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.106429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.106557 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.106650 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.106786 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.106887 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.107033 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.107128 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.107254 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.107409 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.107503 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.107578 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.107779 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.107871 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.108261 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.108356 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.108435 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] publish_errors = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.108510 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] publish_errors = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.108655 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.108752 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.108878 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.108971 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.109058 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.109151 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.109301 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.109446 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.109517 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.109581 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.109751 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] standard_threads = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.109847 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] standard_threads = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.109977 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.110053 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.110215 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.110299 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.110391 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.110475 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.110553 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] use_journal = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.110622 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] use_journal = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.110723 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] use_json = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.110801 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] use_json = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.110880 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] use_stderr = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.110963 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] use_stderr = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.111041 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] use_syslog = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.111117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] use_syslog = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.111269 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.111366 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 21:48:32.111551 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.111639 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.111788 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.111864 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.112041 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.112116 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.112279 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.112355 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.112523 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.112599 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.112757 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.112833 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.113002 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.113082 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.113234 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.113311 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.113479 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.113560 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.113720 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.113796 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.113964 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.114046 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.114196 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.114272 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.114436 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.114513 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.114669 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.114841 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.115635 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.115763 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.115961 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.116087 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.116202 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.116401 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.116479 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.116663 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.116752 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.116921 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.116998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.117165 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.117242 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.117399 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.117486 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.117652 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.117734 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.117910 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.117991 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.118158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.118242 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.118389 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.118465 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.118622 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.118706 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.118850 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.118934 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.119079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.119155 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.119312 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.119423 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.119583 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.119738 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.119819 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.119889 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.120030 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.120107 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.120259 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.120335 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.120503 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.120580 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.120739 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.120815 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.120972 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.121054 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.121205 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.121281 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.121430 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.121511 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.121665 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.121742 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.121895 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.121979 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.122121 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.122196 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.122349 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.122426 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.122574 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.122650 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.122802 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.122880 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.123037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.123114 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.123266 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.123360 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.123526 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.123633 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.123758 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.123867 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.123984 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.124086 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.124217 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.124320 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.124447 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.124573 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.124687 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.124813 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.124890 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.125004 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.125130 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.125211 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.125362 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.125441 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.125595 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.125678 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.125810 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.125925 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.126046 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.126124 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.126274 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.126354 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.126519 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.126607 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.126749 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.126833 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.126983 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.127062 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.127213 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.127304 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.127491 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.127579 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.127723 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.127801 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.127954 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.128036 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.128190 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.128271 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.128421 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.128500 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.128682 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.128799 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.128915 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.128995 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.129159 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.129238 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.129394 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.129479 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.129625 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.129703 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.129858 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.129935 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.130089 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.130165 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.130344 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.130424 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.130591 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.130672 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.130834 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.131086 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.131168 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.131400 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.131483 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.131642 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.131723 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.131879 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.131959 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.132114 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.132192 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.132348 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.132425 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.132581 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.132660 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.132841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.132918 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.133077 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.133155 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.133313 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.133391 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.133548 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.133624 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.133783 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.133861 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.134014 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.134100 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.134243 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.134326 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.134472 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.134549 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.134701 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.134786 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.134927 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.135005 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.135153 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.135232 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.135409 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.135495 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.135640 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.135725 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.135878 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.135956 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.136111 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.136194 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.136355 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.136433 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.136588 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.136666 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.136839 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.136916 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.137068 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] auth.external = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.137146 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.137300 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.137386 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.137536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] auth.external = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.137620 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.137764 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.137869 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.138040 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] auth.password = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.138165 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.138272 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] auth.token = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.138404 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.138511 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.138639 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] auth.password = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.138748 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.138872 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] auth.token = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.138996 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.139115 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.139229 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.139374 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.139502 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.139626 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.139753 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.139860 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.139986 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.140108 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.140224 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] credential.caching = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.140357 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.140473 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.140590 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.140721 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.140828 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] credential.caching = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.140971 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.141077 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.141209 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.141325 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.141454 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.141573 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.141690 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.141809 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.141957 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.142036 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.142211 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.142301 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.142450 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.142530 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.142704 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.142784 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.142984 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.143069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.143230 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.143308 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.143518 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.143597 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.143764 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.143857 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.144021 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.144100 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.144256 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] federation.caching = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.144335 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.144499 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.144586 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.144732 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] federation.caching = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.144809 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.144968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.145047 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.145199 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.145347 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.145443 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.145521 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.145673 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.145752 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.145918 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.146026 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.146186 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.146271 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.146445 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.146682 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.146760 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.146925 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.147003 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.147158 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] identity.caching = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.147236 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.147430 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.147514 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.147677 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.147757 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.147911 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] identity.caching = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.147989 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.148145 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.148223 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.148393 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.148629 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.148865 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.149111 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.149341 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.149581 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.149828 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.149906 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.150074 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.150153 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.150350 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.150446 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.150520 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.150590 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.150747 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.150826 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.150988 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.151076 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.151254 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.151355 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.151506 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.151585 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.151759 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.151850 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.152004 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.152083 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.152262 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.152339 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.152495 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.152580 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.152730 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.152808 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.152958 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.153035 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.153191 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.153277 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.153418 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.153496 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.153651 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.153729 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.153889 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.153975 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.154126 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.154205 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.154364 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.154442 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.154589 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.154739 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.154818 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.154889 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.155063 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.155141 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.155293 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.155430 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.155559 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.155639 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.155803 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.155890 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.156030 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.156120 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.156262 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.156341 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.156489 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.156567 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.156722 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.156812 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.156955 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.157039 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.157185 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.157263 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.157417 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.157502 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.157650 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.157734 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.157888 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.157966 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.158125 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.158212 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.158352 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.158429 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.158580 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.158657 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.158823 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.158910 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.159054 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.159132 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.159286 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.159381 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.159551 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.159641 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.159784 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.159862 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.160010 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.user = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.160088 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.160248 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.160333 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.160486 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.160564 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.160718 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.user = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.160802 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.160959 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.161045 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.161203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.161281 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.161440 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.161518 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.162199 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.162322 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.162452 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.162564 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.162702 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.163007 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.163122 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.163257 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.163382 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.163532 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.163611 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.163767 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.163858 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.164016 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.164096 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.164263 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.164341 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.164508 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.164595 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.164752 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.164831 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.164996 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.165075 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.165227 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.165318 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.165471 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.165550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.165702 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.165779 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.165935 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.166021 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.166170 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.166248 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.166402 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.166479 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.166639 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.166726 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.166881 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.166960 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.167113 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.167190 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.167383 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.167477 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.167614 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.167692 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.167857 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.167936 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.168088 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.168174 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.168319 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.168396 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.168552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.168631 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.168788 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.168874 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.169025 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.169102 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.169255 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.169333 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.169490 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.169642 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.169718 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.169788 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] resource.caching = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.169944 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.170024 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.170186 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] resource.caching = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.170276 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.170423 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.170500 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.170671 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.170749 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.170907 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.170992 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.171139 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.171218 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.171402 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.171482 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.171640 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.171730 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.171877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.171956 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.172108 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] role.caching = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.172186 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.172346 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.172431 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] role.driver = sql {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.172572 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.172649 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] role.caching = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.172814 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.172892 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] role.driver = sql {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.173043 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.173129 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.173288 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.173365 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.173531 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.173608 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.173774 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.173862 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.174017 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.174096 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.174259 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.174337 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.174503 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.174582 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.174730 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.174806 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.174985 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.175063 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.175216 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.175294 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.175485 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.175564 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.175729 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.175807 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.175989 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.176124 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.176237 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.176362 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.176612 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.176689 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.176861 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.176941 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.177109 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.177188 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.177412 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.177494 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.177718 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.177941 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.178054 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.178192 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.178271 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.178444 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.178533 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.178682 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.178759 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.178914 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.178992 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.179143 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.179229 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.179467 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.179549 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.179702 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.179780 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.179955 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.180047 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.180203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.180280 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.180440 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.180524 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.180674 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.180753 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.180903 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.180980 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.181131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.181208 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] token.caching = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.181365 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.181444 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] token.caching = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.181599 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.181677 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.181832 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.181909 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.182071 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.182150 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.182316 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.182392 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.182551 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.182629 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.182786 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.182865 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.183024 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.183101 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.183262 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.183455 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.183545 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.183618 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.183756 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.183835 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.183989 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.184068 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.184221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.184299 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.184466 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.184543 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.184694 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.184772 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.184927 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.185006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.185170 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.185248 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.185459 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] database.connection = **** {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.185548 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.185699 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.185779 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] database.connection = **** {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.185942 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.186033 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.186175 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.186254 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.186409 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.186486 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.186640 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.186725 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.186873 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.186952 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.187105 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.187184 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.187345 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.187436 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.187594 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.187674 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.187824 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.187901 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.188061 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.188146 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.188290 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.188368 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.188523 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.188600 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.188755 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.188904 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.188980 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.189049 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.189224 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.189301 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.189452 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.189537 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.189685 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.189763 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.189854 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 07 21:48:32.189933 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 21:48:32.190075 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 07 21:48:32.220043 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.220269 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.220491 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.220961 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.221458 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.221677 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.229693 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.230225 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.258798 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.258932 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.259019 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.259097 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.311631 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.312218 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.360778 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.361010 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.362278 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.362511 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.426926 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.427154 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.428043 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.428266 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.428479 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.428659 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.428753 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.428912 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.429137 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.429353 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.429567 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.429779 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.451726 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.451942 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.452028 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.452161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.452244 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.452376 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.452503 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.452597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.452679 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.452783 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.452897 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.452982 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.453067 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.453198 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.464246 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.464909 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.477330 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.477594 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.477690 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.477792 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.477877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.478011 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.478104 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.478224 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.478308 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.478434 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.478517 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.478643 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.478736 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.478894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 21:48:32.496728 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.496937 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 07 21:48:32.497195 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.497293 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.497404 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 07 21:48:32.497487 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 07 21:48:32.497700 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.497781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.497923 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 07 21:48:32.498013 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.498187 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.498268 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.498435 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.498515 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.498680 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.498927 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.509223 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.509495 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.509766 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.510044 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.510139 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.510313 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.510576 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.510854 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.518264 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 21:48:32.519043 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 21:48:32.522090 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 21:48:32.522363 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.522640 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.522917 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.523146 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 21:48:32.523231 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.523445 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.523531 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.523706 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.523788 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.523970 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.524053 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.524232 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.524314 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.524491 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.524580 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.524748 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.524829 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.525009 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.525272 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.525539 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.525795 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.550921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.553110 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 21:48:32.553383 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.553480 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.555058 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 21:48:32.555345 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.558729 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 21:48:32.559003 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.559275 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.560706 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 21:48:32.560980 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.561255 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.566541 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 21:48:32.566783 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.568847 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 21:48:32.569125 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.571404 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 21:48:32.571690 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.571965 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.572242 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.572512 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.572781 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.573044 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.573412 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.573570 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 21:48:32.573849 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.574125 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.574407 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.574678 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.574942 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.575201 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.575594 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.592597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.592884 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.595095 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.595457 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.597745 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 21:48:32.600750 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 21:48:32.601664 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.601957 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.604567 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.604863 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.606959 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 21:48:32.609850 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 21:48:32.610570 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 21:48:32.613432 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 21:48:32.614194 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.614489 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.615367 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.615691 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.615975 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.616266 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.617973 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.618264 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.618539 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.618825 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.619106 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.619412 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.635377 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 21:48:32.635659 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.635921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.636189 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.638721 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 21:48:32.639008 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.639279 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.639565 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.644027 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.647960 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.648236 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.648426 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.650266 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.650537 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.651257 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 21:48:32.651561 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.651850 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.652130 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.652412 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.653585 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 21:48:32.653860 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.654134 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.654406 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.654688 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.665778 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 21:48:32.666058 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.666336 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.666612 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.666889 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.667160 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.667458 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.669433 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 21:48:32.669711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.669992 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.670271 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.670557 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.670831 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.671103 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.685736 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 21:48:32.688972 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 21:48:32.689685 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 21:48:32.692441 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.692993 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 21:48:32.694299 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.695940 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 21:48:32.696216 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.696501 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.696684 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.698292 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.699936 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 21:48:32.700208 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.700481 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.703748 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.705338 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 21:48:32.707845 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.709611 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.709937 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.710293 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.710413 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 21:48:32.710568 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.713777 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.714063 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.714337 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.714604 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.719647 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 21:48:32.719927 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.720199 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.723939 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 21:48:32.725191 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.725479 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.729182 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 21:48:32.729462 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.729739 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.730010 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.730278 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.730554 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.730831 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.731100 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.731385 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.731650 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.731923 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.732185 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.732452 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.732710 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.733434 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 21:48:32.733709 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.733985 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.734254 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.734518 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.734796 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.735061 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.735341 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.735621 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.735887 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.736160 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.736423 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.736681 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.736932 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 21:48:32.776956 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5621ec1559c0 pid: 80446 (default app) Jun 07 21:48:32.777167 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5621ec1559c0 pid: 80445 (default app) Jun 07 21:48:32.780477 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a41b30-f5f1-48c2-a281-5d198b060f61 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:32.780771 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a41b30-f5f1-48c2-a281-5d198b060f61 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:32.781047 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a41b30-f5f1-48c2-a281-5d198b060f61 None None] PATH_INFO: `/v3` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:32.785233 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1/1] 10.222.0.13 () {50 vars in 791 bytes} [Tue Jun 7 21:48:32 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:48:40.796249 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37c8fc-a507-432e-980c-5a750f200ea4 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:40.796762 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37c8fc-a507-432e-980c-5a750f200ea4 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:40.796855 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37c8fc-a507-432e-980c-5a750f200ea4 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:40.801068 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1/2] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:48:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:48:40.807413 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b2a91-9942-4f1c-8c9d-206cb534d9cb None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:40.807803 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b2a91-9942-4f1c-8c9d-206cb534d9cb None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:40.808035 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b2a91-9942-4f1c-8c9d-206cb534d9cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:40.875759 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: INFO dbcounter [None req-be6b2a91-9942-4f1c-8c9d-206cb534d9cb None None] Registered counter for database keystone Jun 07 21:48:40.897318 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG oslo_db.sqlalchemy.engines [None req-be6b2a91-9942-4f1c-8c9d-206cb534d9cb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80446) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 07 21:48:40.903626 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writer thread running {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 07 21:48:41.190317 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG passlib.handlers.bcrypt [None req-be6b2a91-9942-4f1c-8c9d-206cb534d9cb None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80446) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 07 21:48:41.190745 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG passlib.handlers.bcrypt [None req-be6b2a91-9942-4f1c-8c9d-206cb534d9cb None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80446) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 07 21:48:41.246293 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-be6b2a91-9942-4f1c-8c9d-206cb534d9cb None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:48:41.247940 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-be6b2a91-9942-4f1c-8c9d-206cb534d9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:48:41.373439 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 2/3] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:48:40 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 567 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 07 21:48:41.553095 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37c8fc-a507-432e-980c-5a750f200ea4 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:48:41.620994 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: INFO dbcounter [None req-6e37c8fc-a507-432e-980c-5a750f200ea4 None None] Registered counter for database keystone Jun 07 21:48:41.640569 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG oslo_db.sqlalchemy.engines [None req-6e37c8fc-a507-432e-980c-5a750f200ea4 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80445) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 07 21:48:41.646686 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writer thread running {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 07 21:48:41.876052 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37c8fc-a507-432e-980c-5a750f200ea4 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:48:41.880562 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f877ee6f-ef91-475b-8ca9-8e004b9c4489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:48:41.881337 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877ee6f-ef91-475b-8ca9-8e004b9c4489 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:41.881624 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877ee6f-ef91-475b-8ca9-8e004b9c4489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:41.881900 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877ee6f-ef91-475b-8ca9-8e004b9c4489 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:41.890639 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f877ee6f-ef91-475b-8ca9-8e004b9c4489 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:48:41.944792 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f877ee6f-ef91-475b-8ca9-8e004b9c4489 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 07 21:48:41.945129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f877ee6f-ef91-475b-8ca9-8e004b9c4489 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:48:41.955604 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 2/4] 10.222.0.13 () {58 vars in 1324 bytes} [Tue Jun 7 21:48:41 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 07 21:48:41.962222 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6b2a91-9942-4f1c-8c9d-206cb534d9cb None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:48:41.983434 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6b2a91-9942-4f1c-8c9d-206cb534d9cb None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:48:41.989409 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6f04c4-a0c7-449f-a1e0-60cc99fde5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:48:41.991194 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f04c4-a0c7-449f-a1e0-60cc99fde5bd admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:41.991547 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f04c4-a0c7-449f-a1e0-60cc99fde5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:41.991832 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f04c4-a0c7-449f-a1e0-60cc99fde5bd admin admin] PATH_INFO: `/v3/projects/c61211fad6e447b0bf162f99f1111a82` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:41.993472 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6f04c4-a0c7-449f-a1e0-60cc99fde5bd admin admin] RBAC: Authorizing `identity:get_project(project_id=c61211fad6e447b0bf162f99f1111a82)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:48:42.061670 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: WARNING keystone.common.rbac_enforcer.enforcer [None req-8f6f04c4-a0c7-449f-a1e0-60cc99fde5bd admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 07 21:48:42.062215 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6f04c4-a0c7-449f-a1e0-60cc99fde5bd admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:48:42.069622 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 3/5] 10.222.0.13 () {56 vars in 1207 bytes} [Tue Jun 7 21:48:41 2022] GET /identity/v3/projects/c61211fad6e447b0bf162f99f1111a82 => generated 344 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:48:44.405950 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a00c26-b981-4a7d-b7e0-d556e64c5f89 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:44.406505 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a00c26-b981-4a7d-b7e0-d556e64c5f89 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:44.406505 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a00c26-b981-4a7d-b7e0-d556e64c5f89 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:44.410677 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 3/6] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:48:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:48:44.418510 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f872f94-716a-4e62-8bb6-f628c1934f64 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:44.418961 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f872f94-716a-4e62-8bb6-f628c1934f64 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:44.419804 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f872f94-716a-4e62-8bb6-f628c1934f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:44.446609 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-0f872f94-716a-4e62-8bb6-f628c1934f64 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:48:44.448970 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-0f872f94-716a-4e62-8bb6-f628c1934f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:48:44.460523 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 4/7] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:48:44 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 07 21:48:44.485934 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a00c26-b981-4a7d-b7e0-d556e64c5f89 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:48:44.498171 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a00c26-b981-4a7d-b7e0-d556e64c5f89 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:48:44.502341 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e91058-7cfe-405e-83db-4b06de12c9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:48:44.502975 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e91058-7cfe-405e-83db-4b06de12c9d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:44.503262 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e91058-7cfe-405e-83db-4b06de12c9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:44.503568 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e91058-7cfe-405e-83db-4b06de12c9d4 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:44.511081 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e91058-7cfe-405e-83db-4b06de12c9d4 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:48:44.512883 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e91058-7cfe-405e-83db-4b06de12c9d4 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:48:44.522088 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 4/8] 10.222.0.13 () {58 vars in 1324 bytes} [Tue Jun 7 21:48:44 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 21:48:44.527676 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f872f94-716a-4e62-8bb6-f628c1934f64 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:48:44.537474 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f872f94-716a-4e62-8bb6-f628c1934f64 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:48:44.542108 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43ecfca-0e46-4307-b70d-e575a89a4cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:48:44.542981 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ecfca-0e46-4307-b70d-e575a89a4cce admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:44.543261 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ecfca-0e46-4307-b70d-e575a89a4cce admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:44.543571 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ecfca-0e46-4307-b70d-e575a89a4cce admin admin] PATH_INFO: `/v3/users/c9d023b1a9bd4c969261ea4adeb6b556` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:44.544991 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43ecfca-0e46-4307-b70d-e575a89a4cce admin admin] RBAC: Authorizing `identity:get_user(user_id=c9d023b1a9bd4c969261ea4adeb6b556)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:48:44.546813 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43ecfca-0e46-4307-b70d-e575a89a4cce admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:48:44.548402 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 5/9] 10.222.0.13 () {56 vars in 1198 bytes} [Tue Jun 7 21:48:44 2022] GET /identity/v3/users/c9d023b1a9bd4c969261ea4adeb6b556 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:48:46.857288 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801f936-57de-4aa2-b233-abdc866e1730 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:46.857825 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801f936-57de-4aa2-b233-abdc866e1730 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:46.857927 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801f936-57de-4aa2-b233-abdc866e1730 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:46.859271 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 5/10] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:48:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:48:46.865037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6dba80-93a4-47b9-945a-a82dd06a7188 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:46.865440 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6dba80-93a4-47b9-945a-a82dd06a7188 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:46.865603 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6dba80-93a4-47b9-945a-a82dd06a7188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:46.882503 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-ae6dba80-93a4-47b9-945a-a82dd06a7188 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:48:46.883703 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-ae6dba80-93a4-47b9-945a-a82dd06a7188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:48:46.903116 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba58d6a-8908-4a4b-a480-7d3189b1cbfa None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:46.903434 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba58d6a-8908-4a4b-a480-7d3189b1cbfa None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:46.903719 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba58d6a-8908-4a4b-a480-7d3189b1cbfa None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:46.905053 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 6/11] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:48:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 21:48:46.910198 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea612bfa-30a4-4e9c-9b25-48c0b773c91c None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:46.910486 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea612bfa-30a4-4e9c-9b25-48c0b773c91c None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:46.910764 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea612bfa-30a4-4e9c-9b25-48c0b773c91c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:46.950519 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG passlib.handlers.bcrypt [None req-ea612bfa-30a4-4e9c-9b25-48c0b773c91c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80445) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 07 21:48:46.950950 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG passlib.handlers.bcrypt [None req-ea612bfa-30a4-4e9c-9b25-48c0b773c91c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80445) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 07 21:48:46.963089 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 6/12] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:48:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 21:48:46.986518 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6dba80-93a4-47b9-945a-a82dd06a7188 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:48:46.998206 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-ea612bfa-30a4-4e9c-9b25-48c0b773c91c None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:48:46.999842 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-ea612bfa-30a4-4e9c-9b25-48c0b773c91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:48:47.010843 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:48:47.089319 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6dba80-93a4-47b9-945a-a82dd06a7188 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:48:47.100663 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 7/13] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:48:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 21:48:47.122621 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea612bfa-30a4-4e9c-9b25-48c0b773c91c None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:48:47.123130 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4958ae-383e-4a0a-a5a2-87f72a3b9e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:48:47.123952 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4958ae-383e-4a0a-a5a2-87f72a3b9e04 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:47.124233 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4958ae-383e-4a0a-a5a2-87f72a3b9e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:47.124508 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4958ae-383e-4a0a-a5a2-87f72a3b9e04 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:47.135698 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4958ae-383e-4a0a-a5a2-87f72a3b9e04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:48:47.146739 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-bc4958ae-383e-4a0a-a5a2-87f72a3b9e04 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:48:47.146739 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:48:47.146739 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:48:47.146739 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:48:47.146739 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:48:47.146739 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:48:47.146739 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:48:47.146739 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:48:47.146739 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:48:47.146739 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:48:47.146739 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:48:47.146739 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:48:47.146739 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:48:47.146739 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:48:47.146739 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:48:47.146739 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:48:47.147923 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:48:47.147923 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:48:47.147923 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:48:47.147923 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:48:47.147923 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:48:47.147923 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:48:47.147923 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:48:47.147923 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:48:47.147923 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:48:47.147923 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:48:47.147923 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:48:47.147923 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:48:47.147923 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:48:47.147923 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:48:47.147923 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:48:47.147923 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:48:47.148766 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:48:47.148766 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:48:47.148766 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:48:47.148766 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:48:47.148766 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:48:47.148766 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:48:47.148766 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:48:47.148766 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:48:47.148766 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:48:47.148766 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:48:47.148766 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:48:47.148766 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:48:47.148766 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 7/14] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jun 7 21:48:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:48:47.153606 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4958ae-383e-4a0a-a5a2-87f72a3b9e04 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:48:47.227626 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea612bfa-30a4-4e9c-9b25-48c0b773c91c None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:48:47.229898 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:48:47.234680 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:48:47.254586 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4958ae-383e-4a0a-a5a2-87f72a3b9e04 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:48:47.267378 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4cc691-ecb4-4417-bc47-c2b3dfc0a02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:48:47.268239 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4cc691-ecb4-4417-bc47-c2b3dfc0a02e None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:47.268537 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4cc691-ecb4-4417-bc47-c2b3dfc0a02e None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:47.268846 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4cc691-ecb4-4417-bc47-c2b3dfc0a02e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:47.287888 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1893642e-1035-4332-af63-0ba281a7b567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:48:47.288556 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893642e-1035-4332-af63-0ba281a7b567 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:47.288850 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893642e-1035-4332-af63-0ba281a7b567 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:47.289128 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893642e-1035-4332-af63-0ba281a7b567 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:47.289894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1893642e-1035-4332-af63-0ba281a7b567 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:48:47.290204 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1893642e-1035-4332-af63-0ba281a7b567 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:48:47.291648 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1893642e-1035-4332-af63-0ba281a7b567 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:48:47.305077 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 8/15] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:48:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:48:47.310358 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1893642e-1035-4332-af63-0ba281a7b567 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:48:47.338929 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4cc691-ecb4-4417-bc47-c2b3dfc0a02e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:48:47.340504 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4cc691-ecb4-4417-bc47-c2b3dfc0a02e None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:48:47.407093 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1893642e-1035-4332-af63-0ba281a7b567 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:48:47.437393 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 8/16] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:48:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 21:48:47.443014 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4cc691-ecb4-4417-bc47-c2b3dfc0a02e None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:48:47.443519 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641c9cb4-7c29-480c-a970-8846f4c7ffde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:48:47.444400 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c9cb4-7c29-480c-a970-8846f4c7ffde None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:47.444685 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c9cb4-7c29-480c-a970-8846f4c7ffde None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:47.444959 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c9cb4-7c29-480c-a970-8846f4c7ffde None admin] PATH_INFO: `/v3/users/c9d023b1a9bd4c969261ea4adeb6b556` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:47.445783 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:48:47.446170 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641c9cb4-7c29-480c-a970-8846f4c7ffde None admin] RBAC: Authorizing `identity:get_user(user_id=c9d023b1a9bd4c969261ea4adeb6b556)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:48:47.448685 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641c9cb4-7c29-480c-a970-8846f4c7ffde None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:48:47.450306 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 9/17] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:48:47 2022] GET /identity/v3/users/c9d023b1a9bd4c969261ea4adeb6b556 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:48:47.455922 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641c9cb4-7c29-480c-a970-8846f4c7ffde None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:48:47.471610 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:48:47.544982 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4cc691-ecb4-4417-bc47-c2b3dfc0a02e None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:48:47.557171 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641c9cb4-7c29-480c-a970-8846f4c7ffde None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:48:47.578083 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94617032-1b3a-493c-9890-ecffe5284710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:48:47.578874 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94617032-1b3a-493c-9890-ecffe5284710 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:47.579154 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94617032-1b3a-493c-9890-ecffe5284710 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:47.579452 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94617032-1b3a-493c-9890-ecffe5284710 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:47.590403 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c17bfea-d593-40ae-ae81-bf244338f3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:48:47.591197 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c17bfea-d593-40ae-ae81-bf244338f3b9 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:47.591523 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c17bfea-d593-40ae-ae81-bf244338f3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:47.591803 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c17bfea-d593-40ae-ae81-bf244338f3b9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:47.592995 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c17bfea-d593-40ae-ae81-bf244338f3b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:48:47.594462 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c17bfea-d593-40ae-ae81-bf244338f3b9 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:48:47.596561 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 10/18] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:48:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:48:47.598542 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94617032-1b3a-493c-9890-ecffe5284710 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:48:47.600090 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94617032-1b3a-493c-9890-ecffe5284710 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:48:47.602513 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c17bfea-d593-40ae-ae81-bf244338f3b9 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:48:47.621615 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-94617032-1b3a-493c-9890-ecffe5284710 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:48:47.621615 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:48:47.621615 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:48:47.621615 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:48:47.621615 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:48:47.621615 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:48:47.621615 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:48:47.621615 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:48:47.621615 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:48:47.621615 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:48:47.621615 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:48:47.621615 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:48:47.621615 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:48:47.621615 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:48:47.621615 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:48:47.622479 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:48:47.622479 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:48:47.622479 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:48:47.622479 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:48:47.622479 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:48:47.622479 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:48:47.622479 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:48:47.622479 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:48:47.622479 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:48:47.622479 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:48:47.622479 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:48:47.622479 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:48:47.622479 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:48:47.622479 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:48:47.622479 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:48:47.622479 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:48:47.622479 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:48:47.623300 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:48:47.623300 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:48:47.623300 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:48:47.623300 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:48:47.623300 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 9/19] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:48:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:48:47.628546 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94617032-1b3a-493c-9890-ecffe5284710 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:48:47.683409 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:48:47.684005 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:48:47.703632 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c17bfea-d593-40ae-ae81-bf244338f3b9 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:48:47.728208 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94617032-1b3a-493c-9890-ecffe5284710 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:48:47.736611 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3cd7d5-8a45-44f5-8cf7-c96b7f4f6128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:48:47.737270 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3cd7d5-8a45-44f5-8cf7-c96b7f4f6128 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:47.737556 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3cd7d5-8a45-44f5-8cf7-c96b7f4f6128 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:47.737837 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3cd7d5-8a45-44f5-8cf7-c96b7f4f6128 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:47.738680 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3cd7d5-8a45-44f5-8cf7-c96b7f4f6128 None admin] RBAC: Adding query filter params (role.id=6b483758dbd44eb4a78df2ce4603d601, scope.domain.id=default, user.id=c9d023b1a9bd4c969261ea4adeb6b556) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:48:47.738993 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3cd7d5-8a45-44f5-8cf7-c96b7f4f6128 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:48:47.741248 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3cd7d5-8a45-44f5-8cf7-c96b7f4f6128 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:48:47.747881 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 11/20] 10.222.0.13 () {56 vars in 1322 bytes} [Tue Jun 7 21:48:47 2022] GET /identity/v3/role_assignments?user.id=c9d023b1a9bd4c969261ea4adeb6b556&scope.domain.id=default&role.id=6b483758dbd44eb4a78df2ce4603d601 => generated 233 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:48:47.765028 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ebc7a-c4e7-4a70-98b9-7d4f4c02499e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:48:47.765722 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ebc7a-c4e7-4a70-98b9-7d4f4c02499e None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:47.766002 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ebc7a-c4e7-4a70-98b9-7d4f4c02499e None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:47.766334 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ebc7a-c4e7-4a70-98b9-7d4f4c02499e None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:47.767065 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55ebc7a-c4e7-4a70-98b9-7d4f4c02499e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:48:47.767403 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55ebc7a-c4e7-4a70-98b9-7d4f4c02499e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:48:47.768857 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55ebc7a-c4e7-4a70-98b9-7d4f4c02499e None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:48:47.789477 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 10/21] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:48:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:48:50.294247 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4fa7e6-4066-478f-9e86-3b65d298b4cb None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:50.295268 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4fa7e6-4066-478f-9e86-3b65d298b4cb None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:50.295779 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4fa7e6-4066-478f-9e86-3b65d298b4cb None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:50.297730 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 12/22] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:48:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:48:50.303273 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8814c469-3f58-4afc-982a-59e830d10743 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:50.304268 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8814c469-3f58-4afc-982a-59e830d10743 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:50.304755 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8814c469-3f58-4afc-982a-59e830d10743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:50.331582 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-8814c469-3f58-4afc-982a-59e830d10743 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:48:50.333223 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-8814c469-3f58-4afc-982a-59e830d10743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:48:50.425844 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 11/23] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:48:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:48:50.464169 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4fa7e6-4066-478f-9e86-3b65d298b4cb None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:48:50.601790 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4fa7e6-4066-478f-9e86-3b65d298b4cb None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:48:50.653873 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3c3a02-a4bc-4b00-80c0-745887660b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:48:50.655036 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c3a02-a4bc-4b00-80c0-745887660b38 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:50.655568 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c3a02-a4bc-4b00-80c0-745887660b38 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:50.656060 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c3a02-a4bc-4b00-80c0-745887660b38 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:50.672340 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3c3a02-a4bc-4b00-80c0-745887660b38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:48:50.681333 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-7d3c3a02-a4bc-4b00-80c0-745887660b38 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:48:50.681333 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:48:50.681333 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:48:50.681333 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:48:50.681333 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:48:50.681333 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:48:50.681333 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:48:50.681333 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:48:50.681333 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:48:50.681333 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:48:50.681333 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:48:50.681333 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:48:50.681333 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:48:50.681333 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:48:50.681333 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:48:50.681333 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:48:50.682191 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:48:50.682191 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:48:50.682191 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:48:50.682191 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:48:50.682191 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:48:50.682191 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:48:50.682191 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:48:50.682191 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:48:50.682191 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:48:50.682191 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:48:50.682191 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:48:50.682191 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:48:50.682191 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:48:50.682191 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:48:50.682191 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:48:50.682191 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:48:50.682931 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:48:50.682931 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:48:50.682931 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:48:50.682931 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:48:50.682931 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:48:50.682931 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:48:50.682931 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:48:50.682931 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:48:50.682931 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:48:50.682931 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:48:50.682931 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:48:50.682931 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:48:50.685140 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 13/24] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jun 7 21:48:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:48:50.692791 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8814c469-3f58-4afc-982a-59e830d10743 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:48:50.751636 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:48:50.838348 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8814c469-3f58-4afc-982a-59e830d10743 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:48:50.874257 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a578d9ef-2ab9-4a44-8689-2a36168a1442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:48:50.874851 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578d9ef-2ab9-4a44-8689-2a36168a1442 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:50.875134 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578d9ef-2ab9-4a44-8689-2a36168a1442 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:50.875455 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578d9ef-2ab9-4a44-8689-2a36168a1442 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:50.876283 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a578d9ef-2ab9-4a44-8689-2a36168a1442 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:48:50.876589 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a578d9ef-2ab9-4a44-8689-2a36168a1442 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:48:50.878032 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a578d9ef-2ab9-4a44-8689-2a36168a1442 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:48:50.893097 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 12/25] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:48:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:48:50.898974 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3c3a02-a4bc-4b00-80c0-745887660b38 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:48:50.906925 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:48:50.930280 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea96c7-7f31-440f-a97f-237e0f02b2a8 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:50.930582 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea96c7-7f31-440f-a97f-237e0f02b2a8 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:50.930839 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea96c7-7f31-440f-a97f-237e0f02b2a8 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:50.932280 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 13/26] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:48:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 21:48:50.937569 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d633385-0ff1-40b7-b403-d45bcafa99e0 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:50.937859 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d633385-0ff1-40b7-b403-d45bcafa99e0 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:50.938142 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d633385-0ff1-40b7-b403-d45bcafa99e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:50.967603 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-2d633385-0ff1-40b7-b403-d45bcafa99e0 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:48:50.968806 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-2d633385-0ff1-40b7-b403-d45bcafa99e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:48:51.132790 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 14/27] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:48:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 21:48:51.139109 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3c3a02-a4bc-4b00-80c0-745887660b38 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:48:51.168433 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d633385-0ff1-40b7-b403-d45bcafa99e0 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:48:51.202071 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd0282-6a5a-41dc-b58f-9713babeff5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:48:51.202953 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd0282-6a5a-41dc-b58f-9713babeff5d None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:51.203231 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd0282-6a5a-41dc-b58f-9713babeff5d None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:51.203527 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd0282-6a5a-41dc-b58f-9713babeff5d None admin] PATH_INFO: `/v3/users/c9d023b1a9bd4c969261ea4adeb6b556` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:51.205703 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebd0282-6a5a-41dc-b58f-9713babeff5d None admin] RBAC: Authorizing `identity:get_user(user_id=c9d023b1a9bd4c969261ea4adeb6b556)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:48:51.207175 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebd0282-6a5a-41dc-b58f-9713babeff5d None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:48:51.211109 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 14/28] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:48:50 2022] GET /identity/v3/users/c9d023b1a9bd4c969261ea4adeb6b556 => generated 247 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:48:51.212605 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177397eb-7815-46f4-8504-6081a28ab72c None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:51.213763 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177397eb-7815-46f4-8504-6081a28ab72c None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:51.214858 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177397eb-7815-46f4-8504-6081a28ab72c None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:51.216200 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 15/29] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:48:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:48:51.217502 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458df63-3746-4d29-a121-171228968c04 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:51.218067 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458df63-3746-4d29-a121-171228968c04 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:51.218537 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458df63-3746-4d29-a121-171228968c04 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:51.219949 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 16/30] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:48:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:48:51.220817 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5458df63-3746-4d29-a121-171228968c04 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:48:51.236225 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:48:51.312496 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d633385-0ff1-40b7-b403-d45bcafa99e0 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:48:51.331710 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:48:51.358972 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5458df63-3746-4d29-a121-171228968c04 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:48:51.361458 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de39e21b-e6bb-4565-990b-7d4a06fcf8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:48:51.363372 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39e21b-e6bb-4565-990b-7d4a06fcf8f3 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:51.364158 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39e21b-e6bb-4565-990b-7d4a06fcf8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:51.364948 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39e21b-e6bb-4565-990b-7d4a06fcf8f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:51.379973 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de39e21b-e6bb-4565-990b-7d4a06fcf8f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:48:51.381839 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de39e21b-e6bb-4565-990b-7d4a06fcf8f3 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:48:51.402180 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-de39e21b-e6bb-4565-990b-7d4a06fcf8f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:48:51.402180 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:48:51.402180 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:48:51.402180 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:48:51.402180 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:48:51.402180 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:48:51.402180 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:48:51.402180 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:48:51.402180 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:48:51.402180 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:48:51.402180 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:48:51.402180 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:48:51.402180 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:48:51.402180 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:48:51.402180 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:48:51.403011 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:48:51.403011 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:48:51.403011 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:48:51.403011 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:48:51.403011 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:48:51.403011 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:48:51.403011 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:48:51.403011 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:48:51.403011 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:48:51.403011 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:48:51.403011 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:48:51.403011 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:48:51.403011 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:48:51.403011 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:48:51.403011 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:48:51.403011 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:48:51.403011 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:48:51.404203 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:48:51.404203 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:48:51.404203 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:48:51.404203 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:48:51.404405 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6342e608-9dab-40d7-880d-2b692abee391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:48:51.404405 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6342e608-9dab-40d7-880d-2b692abee391 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:51.404405 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6342e608-9dab-40d7-880d-2b692abee391 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:51.404405 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6342e608-9dab-40d7-880d-2b692abee391 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:51.405021 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6342e608-9dab-40d7-880d-2b692abee391 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:48:51.406567 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6342e608-9dab-40d7-880d-2b692abee391 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:48:51.406798 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 15/31] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:48:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:48:51.408520 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34132421-7bb3-4fe0-aaac-ffcc26ccaf2a None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:51.409012 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34132421-7bb3-4fe0-aaac-ffcc26ccaf2a None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:51.409487 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34132421-7bb3-4fe0-aaac-ffcc26ccaf2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:51.413831 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 17/32] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:48:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:48:51.415482 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a0aaa-554f-4fb9-b4b8-6f9c17a7a556 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:51.415774 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a0aaa-554f-4fb9-b4b8-6f9c17a7a556 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:51.416058 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a0aaa-554f-4fb9-b4b8-6f9c17a7a556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:51.428468 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-34132421-7bb3-4fe0-aaac-ffcc26ccaf2a None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:48:51.429929 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-34132421-7bb3-4fe0-aaac-ffcc26ccaf2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:48:51.436733 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-1e3a0aaa-554f-4fb9-b4b8-6f9c17a7a556 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:48:51.438508 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-1e3a0aaa-554f-4fb9-b4b8-6f9c17a7a556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:48:51.505087 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 16/33] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:48:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:48:51.506315 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34132421-7bb3-4fe0-aaac-ffcc26ccaf2a None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:48:51.529654 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:48:51.531029 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 18/34] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:48:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:48:51.535971 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a0aaa-554f-4fb9-b4b8-6f9c17a7a556 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:48:51.633728 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34132421-7bb3-4fe0-aaac-ffcc26ccaf2a None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:48:51.642689 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:48:51.677920 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a0aaa-554f-4fb9-b4b8-6f9c17a7a556 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:48:51.683681 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c227c47-b335-45ea-9f64-33b6fa405827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:48:51.684357 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c227c47-b335-45ea-9f64-33b6fa405827 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:51.684641 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c227c47-b335-45ea-9f64-33b6fa405827 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:51.684922 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c227c47-b335-45ea-9f64-33b6fa405827 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:51.685662 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c227c47-b335-45ea-9f64-33b6fa405827 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:48:51.685971 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c227c47-b335-45ea-9f64-33b6fa405827 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:48:51.687916 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c227c47-b335-45ea-9f64-33b6fa405827 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:48:51.701106 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 17/35] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:48:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:48:51.702210 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c227c47-b335-45ea-9f64-33b6fa405827 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:48:51.727953 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1465b528-9d4b-4b5d-83d2-2bd022948b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:48:51.729149 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465b528-9d4b-4b5d-83d2-2bd022948b1f None admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:51.729821 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465b528-9d4b-4b5d-83d2-2bd022948b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:51.730109 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465b528-9d4b-4b5d-83d2-2bd022948b1f None admin] PATH_INFO: `/v3/domains/default/users/c9d023b1a9bd4c969261ea4adeb6b556/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:51.733104 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1465b528-9d4b-4b5d-83d2-2bd022948b1f None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=c9d023b1a9bd4c969261ea4adeb6b556, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:48:51.735174 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1465b528-9d4b-4b5d-83d2-2bd022948b1f None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:48:51.796933 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:48:51.818363 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c227c47-b335-45ea-9f64-33b6fa405827 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:48:51.853901 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 19/36] 10.222.0.13 () {58 vars in 1361 bytes} [Tue Jun 7 21:48:51 2022] PUT /identity/v3/domains/default/users/c9d023b1a9bd4c969261ea4adeb6b556/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 21:48:51.860898 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b3645-bfb1-45fc-9531-22f2b43e3e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:48:51.861029 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1465b528-9d4b-4b5d-83d2-2bd022948b1f None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:48:51.861133 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b3645-bfb1-45fc-9531-22f2b43e3e9f None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:51.861133 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b3645-bfb1-45fc-9531-22f2b43e3e9f None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:51.861133 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b3645-bfb1-45fc-9531-22f2b43e3e9f None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:51.861133 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9b3645-bfb1-45fc-9531-22f2b43e3e9f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:48:51.861133 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9b3645-bfb1-45fc-9531-22f2b43e3e9f None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:48:51.985442 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1465b528-9d4b-4b5d-83d2-2bd022948b1f None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:48:51.997895 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 18/37] 10.222.0.13 () {60 vars in 1131 bytes} [Tue Jun 7 21:48:51 2022] POST /identity/v3/roles => generated 219 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 21:48:51.998720 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b3645-bfb1-45fc-9531-22f2b43e3e9f None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:48:52.036301 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:48:52.048040 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e825fa35-b329-4e92-9007-4044831c5b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:48:52.048735 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825fa35-b329-4e92-9007-4044831c5b3f None admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:52.049028 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825fa35-b329-4e92-9007-4044831c5b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:52.049317 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825fa35-b329-4e92-9007-4044831c5b3f None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:52.050154 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e825fa35-b329-4e92-9007-4044831c5b3f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:48:52.052587 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e825fa35-b329-4e92-9007-4044831c5b3f None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:48:52.097221 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 20/38] 10.222.0.13 () {60 vars in 1131 bytes} [Tue Jun 7 21:48:51 2022] POST /identity/v3/roles => generated 225 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 21:48:52.098893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17cd35-5525-4d23-9794-1450a3959ab2 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:52.099255 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17cd35-5525-4d23-9794-1450a3959ab2 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:52.099777 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17cd35-5525-4d23-9794-1450a3959ab2 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:52.101307 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 21/39] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:48:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:48:52.108922 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd7d90-0eca-41da-9ef1-25fa75243cff None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:52.109247 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd7d90-0eca-41da-9ef1-25fa75243cff None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:52.109506 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd7d90-0eca-41da-9ef1-25fa75243cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:52.140418 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-01bd7d90-0eca-41da-9ef1-25fa75243cff None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:48:52.142031 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-01bd7d90-0eca-41da-9ef1-25fa75243cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:48:52.185157 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b3645-bfb1-45fc-9531-22f2b43e3e9f None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:48:52.231347 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c99fd94-eecd-452d-b3bc-6476ab473688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:48:52.232053 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99fd94-eecd-452d-b3bc-6476ab473688 None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:52.232472 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99fd94-eecd-452d-b3bc-6476ab473688 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:52.232757 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99fd94-eecd-452d-b3bc-6476ab473688 None admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:52.233766 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c99fd94-eecd-452d-b3bc-6476ab473688 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:48:52.235633 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c99fd94-eecd-452d-b3bc-6476ab473688 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:48:52.250135 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:48:52.277591 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 19/40] 10.222.0.13 () {60 vars in 1141 bytes} [Tue Jun 7 21:48:51 2022] POST /identity/v3/projects => generated 301 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 21:48:52.309638 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 22/41] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:48:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:48:52.355609 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c99fd94-eecd-452d-b3bc-6476ab473688 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:48:52.371004 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9501069e-7890-4525-9759-38c459124b97 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:52.371538 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9501069e-7890-4525-9759-38c459124b97 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:52.371609 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9501069e-7890-4525-9759-38c459124b97 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:52.373088 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 23/42] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:48:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:48:52.378501 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d121c-dd82-4d1c-9cf7-c61f127400c7 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:52.378789 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d121c-dd82-4d1c-9cf7-c61f127400c7 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:52.379074 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d121c-dd82-4d1c-9cf7-c61f127400c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:52.412788 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-e00d121c-dd82-4d1c-9cf7-c61f127400c7 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:48:52.414073 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-e00d121c-dd82-4d1c-9cf7-c61f127400c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:48:52.495747 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c99fd94-eecd-452d-b3bc-6476ab473688 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:48:52.511181 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 24/43] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:48:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 21:48:52.539797 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d121c-dd82-4d1c-9cf7-c61f127400c7 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:48:52.544362 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f1556d-4636-4ea8-b780-cf78ad811564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:48:52.545116 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1556d-4636-4ea8-b780-cf78ad811564 None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:52.545407 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1556d-4636-4ea8-b780-cf78ad811564 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:52.545690 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1556d-4636-4ea8-b780-cf78ad811564 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:52.546522 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f1556d-4636-4ea8-b780-cf78ad811564 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:48:52.547983 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f1556d-4636-4ea8-b780-cf78ad811564 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:48:52.566431 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 20/44] 10.222.0.13 () {60 vars in 1131 bytes} [Tue Jun 7 21:48:52 2022] POST /identity/v3/roles => generated 223 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 21:48:52.611084 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:48:52.680636 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d121c-dd82-4d1c-9cf7-c61f127400c7 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:48:52.719084 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30695e8-3c09-4f0e-82fc-181be8a14477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:48:52.719985 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30695e8-3c09-4f0e-82fc-181be8a14477 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:52.720281 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30695e8-3c09-4f0e-82fc-181be8a14477 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:52.720567 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30695e8-3c09-4f0e-82fc-181be8a14477 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:52.722056 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30695e8-3c09-4f0e-82fc-181be8a14477 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:48:52.723707 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30695e8-3c09-4f0e-82fc-181be8a14477 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:48:52.725759 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 25/45] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:48:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:48:52.731269 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f1556d-4636-4ea8-b780-cf78ad811564 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:48:52.798623 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:48:52.852424 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f1556d-4636-4ea8-b780-cf78ad811564 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:48:52.887700 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78dd63a-b643-4b34-9814-76b8bc357a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:48:52.888371 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78dd63a-b643-4b34-9814-76b8bc357a54 None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:52.888660 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78dd63a-b643-4b34-9814-76b8bc357a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:52.888940 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78dd63a-b643-4b34-9814-76b8bc357a54 None admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:52.889669 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78dd63a-b643-4b34-9814-76b8bc357a54 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:48:52.891138 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78dd63a-b643-4b34-9814-76b8bc357a54 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:48:52.925025 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 21/46] 10.222.0.13 () {60 vars in 1141 bytes} [Tue Jun 7 21:48:52 2022] POST /identity/v3/projects => generated 312 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 21:48:54.434539 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5066f-bcbf-4049-8b58-47d05a021200 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:54.435078 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5066f-bcbf-4049-8b58-47d05a021200 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:54.435078 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5066f-bcbf-4049-8b58-47d05a021200 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:54.436487 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 26/47] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:48:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:48:54.441675 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb79ad-2ba4-4355-91cb-5721f27af564 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:54.442106 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb79ad-2ba4-4355-91cb-5721f27af564 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:54.442461 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb79ad-2ba4-4355-91cb-5721f27af564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:54.460828 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-eafb79ad-2ba4-4355-91cb-5721f27af564 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:48:54.462327 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-eafb79ad-2ba4-4355-91cb-5721f27af564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:48:54.527672 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 22/48] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:48:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:48:54.550512 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e5066f-bcbf-4049-8b58-47d05a021200 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:48:54.666642 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e5066f-bcbf-4049-8b58-47d05a021200 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:48:54.683687 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:48:54.700890 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f04735e-06a4-44c8-a06d-ba9e74afe9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:48:54.701621 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f04735e-06a4-44c8-a06d-ba9e74afe9ef None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:54.701909 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f04735e-06a4-44c8-a06d-ba9e74afe9ef None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:54.702186 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f04735e-06a4-44c8-a06d-ba9e74afe9ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:54.712823 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f04735e-06a4-44c8-a06d-ba9e74afe9ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:48:54.720484 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-3f04735e-06a4-44c8-a06d-ba9e74afe9ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:48:54.720484 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:48:54.720484 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:48:54.720484 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:48:54.720484 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:48:54.720484 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:48:54.720484 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:48:54.720484 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:48:54.720484 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:48:54.720484 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:48:54.720484 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:48:54.720484 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:48:54.720484 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:48:54.720484 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:48:54.720484 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:48:54.720484 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:48:54.721738 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:48:54.721738 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:48:54.721738 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:48:54.721738 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:48:54.721738 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:48:54.721738 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:48:54.721738 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:48:54.721738 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:48:54.721738 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:48:54.721738 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:48:54.721738 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:48:54.721738 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:48:54.721738 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:48:54.721738 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:48:54.721738 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:48:54.721738 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:48:54.722509 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:48:54.722509 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:48:54.722509 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:48:54.722509 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:48:54.722509 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:48:54.722509 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:48:54.722509 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:48:54.722509 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:48:54.722509 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:48:54.722509 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:48:54.722509 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:48:54.722509 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:48:54.722509 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 27/49] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jun 7 21:48:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:48:54.727751 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb79ad-2ba4-4355-91cb-5721f27af564 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:48:54.761270 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:48:54.828479 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb79ad-2ba4-4355-91cb-5721f27af564 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:48:54.865116 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddefc234-1a5b-4c32-9cba-53e85e5ee837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:48:54.865833 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddefc234-1a5b-4c32-9cba-53e85e5ee837 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:54.866118 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddefc234-1a5b-4c32-9cba-53e85e5ee837 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:54.866399 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddefc234-1a5b-4c32-9cba-53e85e5ee837 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:54.867626 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddefc234-1a5b-4c32-9cba-53e85e5ee837 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:48:54.868116 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddefc234-1a5b-4c32-9cba-53e85e5ee837 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:48:54.870297 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddefc234-1a5b-4c32-9cba-53e85e5ee837 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:48:54.882942 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 23/50] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:48:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:48:54.891550 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f04735e-06a4-44c8-a06d-ba9e74afe9ef None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:48:54.991371 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f04735e-06a4-44c8-a06d-ba9e74afe9ef None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:48:55.024527 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc4aacd-f225-44ed-ad75-7c7c2fca789c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:48:55.025408 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4aacd-f225-44ed-ad75-7c7c2fca789c None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:55.025695 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4aacd-f225-44ed-ad75-7c7c2fca789c None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:55.025975 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4aacd-f225-44ed-ad75-7c7c2fca789c None admin] PATH_INFO: `/v3/users/c9d023b1a9bd4c969261ea4adeb6b556` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:55.027166 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc4aacd-f225-44ed-ad75-7c7c2fca789c None admin] RBAC: Authorizing `identity:get_user(user_id=c9d023b1a9bd4c969261ea4adeb6b556)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:48:55.028650 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc4aacd-f225-44ed-ad75-7c7c2fca789c None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:48:55.030274 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 28/51] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:48:54 2022] GET /identity/v3/users/c9d023b1a9bd4c969261ea4adeb6b556 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:48:55.035663 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddefc234-1a5b-4c32-9cba-53e85e5ee837 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:48:55.128720 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:48:55.135039 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddefc234-1a5b-4c32-9cba-53e85e5ee837 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:48:55.169772 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e940a77f-ed4b-4fa4-8c11-5903d423fea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:48:55.171001 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e940a77f-ed4b-4fa4-8c11-5903d423fea2 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:55.171396 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e940a77f-ed4b-4fa4-8c11-5903d423fea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:55.171683 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e940a77f-ed4b-4fa4-8c11-5903d423fea2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:55.172878 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e940a77f-ed4b-4fa4-8c11-5903d423fea2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:48:55.174338 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e940a77f-ed4b-4fa4-8c11-5903d423fea2 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:48:55.176000 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 24/52] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:48:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:48:55.180316 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9743dc23-d0df-4289-b197-050683456dd1 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:55.180611 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9743dc23-d0df-4289-b197-050683456dd1 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:55.180895 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9743dc23-d0df-4289-b197-050683456dd1 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:55.182254 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 29/53] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:48:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:48:55.186263 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e940a77f-ed4b-4fa4-8c11-5903d423fea2 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:48:55.187476 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671d41f-b0ed-491b-afd4-08b197901318 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:55.187770 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671d41f-b0ed-491b-afd4-08b197901318 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:55.188052 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671d41f-b0ed-491b-afd4-08b197901318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:55.208733 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-e671d41f-b0ed-491b-afd4-08b197901318 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:48:55.209920 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-e671d41f-b0ed-491b-afd4-08b197901318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:48:55.252240 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:48:55.277294 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 30/54] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:48:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 21:48:55.288856 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e940a77f-ed4b-4fa4-8c11-5903d423fea2 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:48:55.300703 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e671d41f-b0ed-491b-afd4-08b197901318 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:48:55.323023 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac6ffe7-279e-4001-aa9c-48195021602a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:48:55.323707 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6ffe7-279e-4001-aa9c-48195021602a None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:55.323992 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6ffe7-279e-4001-aa9c-48195021602a None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:55.324264 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6ffe7-279e-4001-aa9c-48195021602a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:55.325087 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac6ffe7-279e-4001-aa9c-48195021602a None admin] RBAC: Adding query filter params (role.id=6b483758dbd44eb4a78df2ce4603d601, scope.domain.id=default, user.id=c9d023b1a9bd4c969261ea4adeb6b556) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:48:55.325394 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac6ffe7-279e-4001-aa9c-48195021602a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:48:55.326829 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac6ffe7-279e-4001-aa9c-48195021602a None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:48:55.336321 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 25/55] 10.222.0.13 () {56 vars in 1322 bytes} [Tue Jun 7 21:48:55 2022] GET /identity/v3/role_assignments?user.id=c9d023b1a9bd4c969261ea4adeb6b556&scope.domain.id=default&role.id=6b483758dbd44eb4a78df2ce4603d601 => generated 530 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:48:55.403004 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e671d41f-b0ed-491b-afd4-08b197901318 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:48:55.439642 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2df173-ac03-42d1-a12c-8b66fae0367c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:48:55.440441 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2df173-ac03-42d1-a12c-8b66fae0367c None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:55.440726 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2df173-ac03-42d1-a12c-8b66fae0367c None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:55.441000 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2df173-ac03-42d1-a12c-8b66fae0367c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:55.442237 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2df173-ac03-42d1-a12c-8b66fae0367c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:48:55.443809 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2df173-ac03-42d1-a12c-8b66fae0367c None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:48:55.445509 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 31/56] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:48:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:48:55.451536 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac6ffe7-279e-4001-aa9c-48195021602a None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:48:55.482142 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:48:55.559635 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac6ffe7-279e-4001-aa9c-48195021602a None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:48:55.596631 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfaada6-0916-498f-b1b6-1a6767e211ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:48:55.597302 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfaada6-0916-498f-b1b6-1a6767e211ea None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:55.597583 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfaada6-0916-498f-b1b6-1a6767e211ea None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:55.597855 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfaada6-0916-498f-b1b6-1a6767e211ea None admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:55.599106 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfaada6-0916-498f-b1b6-1a6767e211ea None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:48:55.601178 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfaada6-0916-498f-b1b6-1a6767e211ea None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:48:55.627046 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 26/57] 10.222.0.13 () {60 vars in 1140 bytes} [Tue Jun 7 21:48:55 2022] POST /identity/v3/projects => generated 298 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 21:48:57.889872 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209c4aa-fc42-42d0-8cd4-ba13a930a3a8 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:57.890370 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209c4aa-fc42-42d0-8cd4-ba13a930a3a8 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:57.890456 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209c4aa-fc42-42d0-8cd4-ba13a930a3a8 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:57.891858 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 32/58] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:48:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:48:57.897233 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a483564-7390-48a9-af4a-751e441d120f None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:57.897663 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a483564-7390-48a9-af4a-751e441d120f None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:57.897795 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a483564-7390-48a9-af4a-751e441d120f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:57.914655 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-3a483564-7390-48a9-af4a-751e441d120f None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:48:57.915874 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-3a483564-7390-48a9-af4a-751e441d120f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:48:57.988238 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 27/59] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:48:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:48:58.015723 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2209c4aa-fc42-42d0-8cd4-ba13a930a3a8 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:48:58.062025 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:48:58.115265 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2209c4aa-fc42-42d0-8cd4-ba13a930a3a8 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:48:58.147486 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3676cde6-b5aa-4cf1-a991-00e6937054db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:48:58.148265 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676cde6-b5aa-4cf1-a991-00e6937054db None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:58.148552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676cde6-b5aa-4cf1-a991-00e6937054db None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:58.148830 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676cde6-b5aa-4cf1-a991-00e6937054db None admin] PATH_INFO: `/v3/domains/default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:58.150018 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3676cde6-b5aa-4cf1-a991-00e6937054db None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:48:58.151502 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3676cde6-b5aa-4cf1-a991-00e6937054db None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:48:58.153080 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 33/60] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:48:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:48:58.158423 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a483564-7390-48a9-af4a-751e441d120f None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:48:58.170302 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:48:58.255472 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a483564-7390-48a9-af4a-751e441d120f None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:48:58.289872 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a8bfe1-15f0-482d-854a-43ebdcc0973b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:48:58.290541 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8bfe1-15f0-482d-854a-43ebdcc0973b None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:48:58.290826 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8bfe1-15f0-482d-854a-43ebdcc0973b None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:48:58.291106 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8bfe1-15f0-482d-854a-43ebdcc0973b None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:48:58.291856 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a8bfe1-15f0-482d-854a-43ebdcc0973b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:48:58.293300 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a8bfe1-15f0-482d-854a-43ebdcc0973b None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:48:58.317739 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 28/61] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Jun 7 21:48:58 2022] POST /identity/v3/users => generated 275 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 21:49:01.815463 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33afbd5-42bb-4317-93d1-921e44bead72 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:01.815967 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33afbd5-42bb-4317-93d1-921e44bead72 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:01.816040 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33afbd5-42bb-4317-93d1-921e44bead72 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:01.817709 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 34/62] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:01.823345 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be47789-e485-41f0-aefb-a6cdee6fa07f None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:01.824172 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be47789-e485-41f0-aefb-a6cdee6fa07f None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:01.824655 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be47789-e485-41f0-aefb-a6cdee6fa07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:01.848560 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-4be47789-e485-41f0-aefb-a6cdee6fa07f None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:01.849801 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-4be47789-e485-41f0-aefb-a6cdee6fa07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:01.862801 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111f0e0-455f-4591-9aa0-523a033549ee None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:01.863118 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111f0e0-455f-4591-9aa0-523a033549ee None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:01.863392 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111f0e0-455f-4591-9aa0-523a033549ee None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:01.865379 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 35/63] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:01.871379 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee7b92-5b0e-4449-a867-b858aef06e4f None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:01.871675 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee7b92-5b0e-4449-a867-b858aef06e4f None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:01.871952 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee7b92-5b0e-4449-a867-b858aef06e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:01.904659 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-6fee7b92-5b0e-4449-a867-b858aef06e4f None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:01.905879 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-6fee7b92-5b0e-4449-a867-b858aef06e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:01.975473 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:01.988214 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 29/64] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:49:02.013896 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be47789-e485-41f0-aefb-a6cdee6fa07f None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:02.038805 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 36/65] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 21:49:02.074900 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fee7b92-5b0e-4449-a867-b858aef06e4f None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:02.120372 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be47789-e485-41f0-aefb-a6cdee6fa07f None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:02.127785 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:02.157431 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab96bb-6895-48e4-adf1-bce49a68d344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:02.158457 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab96bb-6895-48e4-adf1-bce49a68d344 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:02.158886 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab96bb-6895-48e4-adf1-bce49a68d344 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:02.159341 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab96bb-6895-48e4-adf1-bce49a68d344 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:02.179356 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ab96bb-6895-48e4-adf1-bce49a68d344 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:02.186210 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-60ab96bb-6895-48e4-adf1-bce49a68d344 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:49:02.186210 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:02.186210 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:02.186210 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:02.186210 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:02.186210 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:02.186210 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:02.186210 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:02.186210 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:02.186210 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:02.186210 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:02.186210 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:02.186210 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:02.186210 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:02.186210 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:02.186210 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:02.187261 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:02.187261 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:02.187261 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:02.187261 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:02.187261 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:02.187261 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:02.187261 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:02.187261 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:02.187261 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:02.187261 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:02.187261 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:02.187261 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:02.187261 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:02.187261 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:02.187261 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:02.187261 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:02.188087 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:02.188087 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:02.188087 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:02.188087 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:02.188087 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:02.188087 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:02.188087 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:02.188087 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:02.188087 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:02.188087 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:02.188087 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:49:02.188087 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:49:02.189971 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 30/66] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jun 7 21:49:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:49:02.191583 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6125a9b-fd42-429d-abb6-76efad2e383e None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:02.192081 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6125a9b-fd42-429d-abb6-76efad2e383e None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:02.192506 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6125a9b-fd42-429d-abb6-76efad2e383e None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:02.194989 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fee7b92-5b0e-4449-a867-b858aef06e4f None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:02.198675 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 31/67] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:02.199673 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6125a9b-fd42-429d-abb6-76efad2e383e None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:02.231587 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f8ebd3-1f97-4deb-a1e0-ec47cde25f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:02.232394 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8ebd3-1f97-4deb-a1e0-ec47cde25f98 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:02.232679 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8ebd3-1f97-4deb-a1e0-ec47cde25f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:02.232947 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8ebd3-1f97-4deb-a1e0-ec47cde25f98 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:02.244450 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f8ebd3-1f97-4deb-a1e0-ec47cde25f98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:02.252332 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-b2f8ebd3-1f97-4deb-a1e0-ec47cde25f98 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 21:49:02.252332 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:02.252332 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:02.252332 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:02.252332 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:02.252332 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:02.252332 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:02.252332 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:02.252332 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:02.252332 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:02.252332 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:02.252332 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:02.252332 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:02.252332 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:02.252332 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:02.252332 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:02.253258 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:02.253258 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:02.253258 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:02.253258 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:02.253258 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:02.253258 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:02.253258 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:02.253258 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:02.253258 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:02.253258 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:02.253258 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:02.253258 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:02.253258 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:02.253258 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:02.253258 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:02.253258 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:02.254011 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:02.254011 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:02.254011 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:02.254011 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:02.254011 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:02.254011 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:02.254011 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:02.254011 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:02.254011 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:02.254011 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:02.254011 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 21:49:02.254011 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:49:02.256549 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 37/68] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:49:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:49:02.257967 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8439d-3e3c-4d3b-85c2-a05ed8751db2 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:02.258433 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8439d-3e3c-4d3b-85c2-a05ed8751db2 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:02.258890 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8439d-3e3c-4d3b-85c2-a05ed8751db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:02.269260 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:02.277428 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-09e8439d-3e3c-4d3b-85c2-a05ed8751db2 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:02.278639 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-09e8439d-3e3c-4d3b-85c2-a05ed8751db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:02.311531 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6125a9b-fd42-429d-abb6-76efad2e383e None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:02.349348 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825f373f-c406-401b-9172-df19d9a94af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:02.350031 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825f373f-c406-401b-9172-df19d9a94af5 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:02.350322 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825f373f-c406-401b-9172-df19d9a94af5 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:02.350479 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 38/69] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:49:02.351337 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e8439d-3e3c-4d3b-85c2-a05ed8751db2 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:02.355637 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825f373f-c406-401b-9172-df19d9a94af5 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:02.356429 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825f373f-c406-401b-9172-df19d9a94af5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:02.356751 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825f373f-c406-401b-9172-df19d9a94af5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:02.358161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825f373f-c406-401b-9172-df19d9a94af5 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:02.367697 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 32/70] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:49:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:02.369255 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a01261c-13c4-43c4-85ff-541efca5006c None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:02.369691 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a01261c-13c4-43c4-85ff-541efca5006c None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:02.370115 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a01261c-13c4-43c4-85ff-541efca5006c None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:02.371827 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 33/71] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:02.372908 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a01261c-13c4-43c4-85ff-541efca5006c None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:02.398227 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:02.471600 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e8439d-3e3c-4d3b-85c2-a05ed8751db2 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:02.483631 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a01261c-13c4-43c4-85ff-541efca5006c None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:02.501954 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:02.506854 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f48f124-d4c3-4895-a97d-dd155b91ba2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:02.507552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48f124-d4c3-4895-a97d-dd155b91ba2d None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:02.507833 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48f124-d4c3-4895-a97d-dd155b91ba2d None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:02.508110 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48f124-d4c3-4895-a97d-dd155b91ba2d None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:02.508942 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f48f124-d4c3-4895-a97d-dd155b91ba2d None admin] RBAC: Adding query filter params (name=member) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:02.509253 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f48f124-d4c3-4895-a97d-dd155b91ba2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:02.510976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f48f124-d4c3-4895-a97d-dd155b91ba2d None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:02.518985 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 39/72] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jun 7 21:49:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:02.519420 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc63de-9995-489e-bead-582a82108af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:02.520719 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79eeb6a-0143-493f-b9c1-1ed0e0498336 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:02.520981 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc63de-9995-489e-bead-582a82108af9 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:02.521120 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79eeb6a-0143-493f-b9c1-1ed0e0498336 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:02.521200 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc63de-9995-489e-bead-582a82108af9 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:02.521395 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79eeb6a-0143-493f-b9c1-1ed0e0498336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:02.521543 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc63de-9995-489e-bead-582a82108af9 None admin] PATH_INFO: `/v3/users/c9d023b1a9bd4c969261ea4adeb6b556` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:02.524561 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fc63de-9995-489e-bead-582a82108af9 None admin] RBAC: Authorizing `identity:get_user(user_id=c9d023b1a9bd4c969261ea4adeb6b556)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:02.526019 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fc63de-9995-489e-bead-582a82108af9 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:02.531802 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 34/73] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:49:02 2022] GET /identity/v3/users/c9d023b1a9bd4c969261ea4adeb6b556 => generated 247 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:02.531802 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc63de-9995-489e-bead-582a82108af9 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:02.539088 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-e79eeb6a-0143-493f-b9c1-1ed0e0498336 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:02.540304 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-e79eeb6a-0143-493f-b9c1-1ed0e0498336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:02.605694 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 40/74] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:49:02.607002 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee7bde6-9d87-4378-b943-7f823bd43bfb None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:02.607295 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee7bde6-9d87-4378-b943-7f823bd43bfb None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:02.607608 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee7bde6-9d87-4378-b943-7f823bd43bfb None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:02.608886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 41/75] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:02.609574 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee7bde6-9d87-4378-b943-7f823bd43bfb None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:02.632858 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc63de-9995-489e-bead-582a82108af9 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:02.647100 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:02.666697 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b202c-c8dc-40b5-94d7-bf0ef6473e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:02.667536 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b202c-c8dc-40b5-94d7-bf0ef6473e71 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:02.667819 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b202c-c8dc-40b5-94d7-bf0ef6473e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:02.668097 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b202c-c8dc-40b5-94d7-bf0ef6473e71 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:02.678585 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7b202c-c8dc-40b5-94d7-bf0ef6473e71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:02.686577 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-1f7b202c-c8dc-40b5-94d7-bf0ef6473e71 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 21:49:02.686577 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:02.686577 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:02.686577 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:02.686577 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:02.686577 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:02.686577 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:02.686577 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:02.686577 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:02.686577 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:02.686577 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:02.686577 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:02.686577 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:02.686577 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:02.686577 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:02.686577 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:02.687621 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:02.687621 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:02.687621 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:02.687621 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:02.687621 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:02.687621 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:02.687621 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:02.687621 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:02.687621 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:02.687621 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:02.687621 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:02.687621 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:02.687621 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:02.687621 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:02.687621 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:02.687621 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:02.688450 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:02.688450 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:02.688450 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:02.688450 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:02.688450 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:02.688450 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:02.688450 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:02.688450 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:02.688450 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:02.688450 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:02.688450 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 21:49:02.688450 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:49:02.688450 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 35/76] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Jun 7 21:49:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:49:02.689079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b202c-c8dc-40b5-94d7-bf0ef6473e71 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:02.727271 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee7bde6-9d87-4378-b943-7f823bd43bfb None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:02.739906 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:02.762434 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01714a75-8fcf-4143-8a58-55d28d9900f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:02.763314 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01714a75-8fcf-4143-8a58-55d28d9900f8 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:02.763630 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01714a75-8fcf-4143-8a58-55d28d9900f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:02.766209 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01714a75-8fcf-4143-8a58-55d28d9900f8 None admin] PATH_INFO: `/v3/users/9b2edac556674677ba6894dc26bd2aab` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:02.777512 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01714a75-8fcf-4143-8a58-55d28d9900f8 None admin] RBAC: Authorizing `identity:get_user(user_id=9b2edac556674677ba6894dc26bd2aab)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:02.778918 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01714a75-8fcf-4143-8a58-55d28d9900f8 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:02.780634 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 42/77] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:49:02 2022] GET /identity/v3/users/9b2edac556674677ba6894dc26bd2aab => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:02.781821 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec815a-904c-498c-b35e-5f73f89dc9bd None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:02.782112 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec815a-904c-498c-b35e-5f73f89dc9bd None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:02.782393 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec815a-904c-498c-b35e-5f73f89dc9bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:02.798617 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b202c-c8dc-40b5-94d7-bf0ef6473e71 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:02.799425 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-afec815a-904c-498c-b35e-5f73f89dc9bd None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:02.800819 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-afec815a-904c-498c-b35e-5f73f89dc9bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:02.838587 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea1f038-7831-4534-819d-e13d008b56e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:02.839237 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1f038-7831-4534-819d-e13d008b56e1 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:02.839549 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1f038-7831-4534-819d-e13d008b56e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:02.839828 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1f038-7831-4534-819d-e13d008b56e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:02.867723 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 43/78] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:49:02.869077 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afec815a-904c-498c-b35e-5f73f89dc9bd None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:02.883204 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:02.913668 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea1f038-7831-4534-819d-e13d008b56e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:02.915124 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea1f038-7831-4534-819d-e13d008b56e1 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:02.960347 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:02.968891 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afec815a-904c-498c-b35e-5f73f89dc9bd None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:03.001986 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b3079-e266-4648-9b69-ebabf8797d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:03.002784 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b3079-e266-4648-9b69-ebabf8797d5e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:03.003069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b3079-e266-4648-9b69-ebabf8797d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:03.003376 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b3079-e266-4648-9b69-ebabf8797d5e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:03.013912 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098b3079-e266-4648-9b69-ebabf8797d5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:03.014788 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 36/79] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:49:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 21:49:03.015660 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea1f038-7831-4534-819d-e13d008b56e1 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:03.022028 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-098b3079-e266-4648-9b69-ebabf8797d5e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 21:49:03.022028 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:03.022028 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:03.022028 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:03.022028 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:03.022028 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:03.022028 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:03.022028 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:03.022028 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:03.022028 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:03.022028 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:03.022028 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:03.022028 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:03.022028 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:03.022028 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:03.022028 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:03.022894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:03.022894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:03.022894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:03.022894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:03.022894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:03.022894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:03.022894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:03.022894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:03.022894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:03.022894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:03.022894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:03.022894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:03.022894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:03.022894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:03.022894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:03.022894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:03.023999 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:03.023999 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:03.023999 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:03.023999 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:03.023999 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:03.023999 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:03.023999 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:03.023999 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:03.023999 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:03.023999 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:03.023999 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 21:49:03.023999 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:49:03.023999 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 44/80] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:49:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:49:03.023999 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b3079-e266-4648-9b69-ebabf8797d5e None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:03.093419 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:03.117137 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea1f038-7831-4534-819d-e13d008b56e1 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:03.124149 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b3079-e266-4648-9b69-ebabf8797d5e None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:03.150462 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc61dd-4497-464e-a47a-e9b3b1df359e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:03.151127 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc61dd-4497-464e-a47a-e9b3b1df359e None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:03.151441 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc61dd-4497-464e-a47a-e9b3b1df359e None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:03.151715 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc61dd-4497-464e-a47a-e9b3b1df359e None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:03.152456 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dc61dd-4497-464e-a47a-e9b3b1df359e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:03.152847 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dc61dd-4497-464e-a47a-e9b3b1df359e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:03.154128 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dc61dd-4497-464e-a47a-e9b3b1df359e None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:03.156709 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0351167a-c200-48b8-8cfb-42efaeb4c294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:03.157353 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0351167a-c200-48b8-8cfb-42efaeb4c294 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:03.157644 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0351167a-c200-48b8-8cfb-42efaeb4c294 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:03.157927 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0351167a-c200-48b8-8cfb-42efaeb4c294 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:03.162708 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 37/81] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:49:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:03.163518 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc61dd-4497-464e-a47a-e9b3b1df359e None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:03.174455 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:03.226668 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0351167a-c200-48b8-8cfb-42efaeb4c294 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:03.228181 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0351167a-c200-48b8-8cfb-42efaeb4c294 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:03.263594 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc61dd-4497-464e-a47a-e9b3b1df359e None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:03.294100 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:03.297872 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f1a92-b44b-47e1-bcb9-6a2a4e73785b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:03.298660 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f1a92-b44b-47e1-bcb9-6a2a4e73785b None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:03.298942 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f1a92-b44b-47e1-bcb9-6a2a4e73785b None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:03.299635 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f1a92-b44b-47e1-bcb9-6a2a4e73785b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:03.300824 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055f1a92-b44b-47e1-bcb9-6a2a4e73785b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:03.303161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055f1a92-b44b-47e1-bcb9-6a2a4e73785b None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:03.305098 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 38/82] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:49:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:03.306137 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f1a92-b44b-47e1-bcb9-6a2a4e73785b None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:03.330077 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 45/83] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:49:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 21:49:03.330917 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0351167a-c200-48b8-8cfb-42efaeb4c294 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:03.384323 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:03.406455 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f1a92-b44b-47e1-bcb9-6a2a4e73785b None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:03.427827 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0351167a-c200-48b8-8cfb-42efaeb4c294 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:03.441065 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca5bbb-efad-4516-8c0f-512c36e12ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:03.441924 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca5bbb-efad-4516-8c0f-512c36e12ccc None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:03.442362 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca5bbb-efad-4516-8c0f-512c36e12ccc None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:03.442642 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca5bbb-efad-4516-8c0f-512c36e12ccc None admin] PATH_INFO: `/v3/projects/b1e275c904b64687b29e7ca3d28fdda7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:03.443857 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ca5bbb-efad-4516-8c0f-512c36e12ccc None admin] RBAC: Authorizing `identity:get_project(project_id=b1e275c904b64687b29e7ca3d28fdda7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:03.445285 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ca5bbb-efad-4516-8c0f-512c36e12ccc None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:03.446960 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 39/84] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:49:03 2022] GET /identity/v3/projects/b1e275c904b64687b29e7ca3d28fdda7 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:03.447740 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca5bbb-efad-4516-8c0f-512c36e12ccc None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:03.466009 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d413c2f1-0d20-4f67-82fb-50ca574d8947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:03.466679 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d413c2f1-0d20-4f67-82fb-50ca574d8947 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:03.466969 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d413c2f1-0d20-4f67-82fb-50ca574d8947 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:03.467251 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d413c2f1-0d20-4f67-82fb-50ca574d8947 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:03.468037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d413c2f1-0d20-4f67-82fb-50ca574d8947 None admin] RBAC: Adding query filter params (name=member) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:03.468349 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d413c2f1-0d20-4f67-82fb-50ca574d8947 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:03.469718 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d413c2f1-0d20-4f67-82fb-50ca574d8947 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:03.479723 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 46/85] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jun 7 21:49:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:03.480515 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d413c2f1-0d20-4f67-82fb-50ca574d8947 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:03.514027 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:03.549805 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca5bbb-efad-4516-8c0f-512c36e12ccc None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:03.580988 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d413c2f1-0d20-4f67-82fb-50ca574d8947 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:03.584331 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425204fd-3a2e-4965-8c45-429e4bd39a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:03.585203 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425204fd-3a2e-4965-8c45-429e4bd39a4c None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:03.585489 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425204fd-3a2e-4965-8c45-429e4bd39a4c None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:03.585767 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425204fd-3a2e-4965-8c45-429e4bd39a4c None admin] PATH_INFO: `/v3/users/9b2edac556674677ba6894dc26bd2aab` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:03.587003 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425204fd-3a2e-4965-8c45-429e4bd39a4c None admin] RBAC: Authorizing `identity:get_user(user_id=9b2edac556674677ba6894dc26bd2aab)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:03.588479 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425204fd-3a2e-4965-8c45-429e4bd39a4c None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:03.590098 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 40/86] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:49:03 2022] GET /identity/v3/users/9b2edac556674677ba6894dc26bd2aab => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:03.592428 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425204fd-3a2e-4965-8c45-429e4bd39a4c None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:03.607984 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:03.616429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4a096d-0626-44c9-8c54-94ec4d5e4699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:03.617104 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a096d-0626-44c9-8c54-94ec4d5e4699 None admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:03.617392 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a096d-0626-44c9-8c54-94ec4d5e4699 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:03.617675 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a096d-0626-44c9-8c54-94ec4d5e4699 None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:03.618389 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4a096d-0626-44c9-8c54-94ec4d5e4699 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:03.619866 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4a096d-0626-44c9-8c54-94ec4d5e4699 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:03.646047 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 47/87] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Jun 7 21:49:03 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 21:49:03.646557 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4a096d-0626-44c9-8c54-94ec4d5e4699 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:03.702607 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425204fd-3a2e-4965-8c45-429e4bd39a4c None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:03.747659 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3248b656-a205-4c61-9c0e-484ac14ca2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:03.748536 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248b656-a205-4c61-9c0e-484ac14ca2a6 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:03.748826 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248b656-a205-4c61-9c0e-484ac14ca2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:03.749091 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248b656-a205-4c61-9c0e-484ac14ca2a6 None admin] PATH_INFO: `/v3/projects/b1e275c904b64687b29e7ca3d28fdda7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:03.750386 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3248b656-a205-4c61-9c0e-484ac14ca2a6 None admin] RBAC: Authorizing `identity:get_project(project_id=b1e275c904b64687b29e7ca3d28fdda7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:03.751903 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3248b656-a205-4c61-9c0e-484ac14ca2a6 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:03.759172 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 41/88] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:49:03 2022] GET /identity/v3/projects/b1e275c904b64687b29e7ca3d28fdda7 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:03.759172 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3248b656-a205-4c61-9c0e-484ac14ca2a6 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:03.782181 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:03.795216 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4a096d-0626-44c9-8c54-94ec4d5e4699 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:03.833738 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06665aef-2333-4965-88a1-4c0ffa269eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:03.834403 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06665aef-2333-4965-88a1-4c0ffa269eea None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:03.834682 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06665aef-2333-4965-88a1-4c0ffa269eea None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:03.834956 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06665aef-2333-4965-88a1-4c0ffa269eea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:03.835841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06665aef-2333-4965-88a1-4c0ffa269eea None admin] RBAC: Adding query filter params (role.id=6b483758dbd44eb4a78df2ce4603d601, scope.project.id=b1e275c904b64687b29e7ca3d28fdda7, user.id=c9d023b1a9bd4c969261ea4adeb6b556) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:03.836155 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06665aef-2333-4965-88a1-4c0ffa269eea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:03.837629 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06665aef-2333-4965-88a1-4c0ffa269eea None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:03.844481 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 48/89] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:49:03 2022] GET /identity/v3/role_assignments?user.id=c9d023b1a9bd4c969261ea4adeb6b556&scope.project.id=b1e275c904b64687b29e7ca3d28fdda7&role.id=6b483758dbd44eb4a78df2ce4603d601 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:03.849394 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06665aef-2333-4965-88a1-4c0ffa269eea None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:03.853530 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:03.887091 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3248b656-a205-4c61-9c0e-484ac14ca2a6 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:03.941938 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35404e47-8b31-489e-a3dd-9307da576042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:03.941938 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35404e47-8b31-489e-a3dd-9307da576042 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:03.941938 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35404e47-8b31-489e-a3dd-9307da576042 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:03.942961 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35404e47-8b31-489e-a3dd-9307da576042 None admin] PATH_INFO: `/v3/users/9b2edac556674677ba6894dc26bd2aab` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:03.945443 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35404e47-8b31-489e-a3dd-9307da576042 None admin] RBAC: Authorizing `identity:get_user(user_id=9b2edac556674677ba6894dc26bd2aab)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:03.946102 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35404e47-8b31-489e-a3dd-9307da576042 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:03.948040 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 42/90] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:49:03 2022] GET /identity/v3/users/9b2edac556674677ba6894dc26bd2aab => generated 275 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:03.948958 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35404e47-8b31-489e-a3dd-9307da576042 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:04.003744 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06665aef-2333-4965-88a1-4c0ffa269eea None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:04.058380 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf330ac-d50c-4413-8bf6-23ea7d06eb44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:04.059036 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf330ac-d50c-4413-8bf6-23ea7d06eb44 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:04.059379 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf330ac-d50c-4413-8bf6-23ea7d06eb44 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:04.059625 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf330ac-d50c-4413-8bf6-23ea7d06eb44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:04.091753 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35404e47-8b31-489e-a3dd-9307da576042 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:04.102156 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:04.144753 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b550fbe6-b689-45d6-bb23-14d9ca4d2ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:04.146113 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b550fbe6-b689-45d6-bb23-14d9ca4d2ff8 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:04.146592 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b550fbe6-b689-45d6-bb23-14d9ca4d2ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:04.146877 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b550fbe6-b689-45d6-bb23-14d9ca4d2ff8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:04.148205 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b550fbe6-b689-45d6-bb23-14d9ca4d2ff8 None admin] RBAC: Adding query filter params (role.id=c5431c9deadf47e496264ba5b78eeaf6, scope.project.id=b1e275c904b64687b29e7ca3d28fdda7, user.id=9b2edac556674677ba6894dc26bd2aab) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:04.148674 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b550fbe6-b689-45d6-bb23-14d9ca4d2ff8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:04.150563 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b550fbe6-b689-45d6-bb23-14d9ca4d2ff8 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:04.158769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 43/91] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:49:03 2022] GET /identity/v3/role_assignments?user.id=9b2edac556674677ba6894dc26bd2aab&scope.project.id=b1e275c904b64687b29e7ca3d28fdda7&role.id=c5431c9deadf47e496264ba5b78eeaf6 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:04.159658 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b550fbe6-b689-45d6-bb23-14d9ca4d2ff8 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:04.173910 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf330ac-d50c-4413-8bf6-23ea7d06eb44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:04.175525 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf330ac-d50c-4413-8bf6-23ea7d06eb44 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:04.177808 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:04.283520 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b550fbe6-b689-45d6-bb23-14d9ca4d2ff8 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:04.322873 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba751d57-bc2f-46c8-9197-441f655ada0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:04.323782 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba751d57-bc2f-46c8-9197-441f655ada0d None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:04.324149 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba751d57-bc2f-46c8-9197-441f655ada0d None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:04.324625 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba751d57-bc2f-46c8-9197-441f655ada0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:04.331028 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 49/92] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:49:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 21:49:04.337882 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf330ac-d50c-4413-8bf6-23ea7d06eb44 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:04.408005 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba751d57-bc2f-46c8-9197-441f655ada0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:04.409721 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba751d57-bc2f-46c8-9197-441f655ada0d None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:04.427887 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:04.446622 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:04.494964 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf330ac-d50c-4413-8bf6-23ea7d06eb44 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:04.535556 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd88f9a2-86b5-43b4-a54c-7b30042c7097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:04.535556 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd88f9a2-86b5-43b4-a54c-7b30042c7097 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:04.535556 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd88f9a2-86b5-43b4-a54c-7b30042c7097 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:04.535556 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd88f9a2-86b5-43b4-a54c-7b30042c7097 None admin] PATH_INFO: `/v3/projects/b1e275c904b64687b29e7ca3d28fdda7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:04.536287 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd88f9a2-86b5-43b4-a54c-7b30042c7097 None admin] RBAC: Authorizing `identity:get_project(project_id=b1e275c904b64687b29e7ca3d28fdda7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:04.536287 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd88f9a2-86b5-43b4-a54c-7b30042c7097 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:04.541107 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 50/93] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:49:04 2022] GET /identity/v3/projects/b1e275c904b64687b29e7ca3d28fdda7 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:04.542401 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd88f9a2-86b5-43b4-a54c-7b30042c7097 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:04.542894 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 44/94] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:49:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 21:49:04.548393 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba751d57-bc2f-46c8-9197-441f655ada0d None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:04.647696 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd88f9a2-86b5-43b4-a54c-7b30042c7097 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:04.679852 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:04.688940 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db23f8b-4b73-478f-87cf-a987ebe2aa75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:04.689607 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db23f8b-4b73-478f-87cf-a987ebe2aa75 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:04.689889 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db23f8b-4b73-478f-87cf-a987ebe2aa75 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:04.690164 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db23f8b-4b73-478f-87cf-a987ebe2aa75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:04.691089 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db23f8b-4b73-478f-87cf-a987ebe2aa75 None admin] RBAC: Adding query filter params (role.id=50e04e9d09e045b0bcf8e1782a6e2f23, scope.project.id=b1e275c904b64687b29e7ca3d28fdda7, user.id=9b2edac556674677ba6894dc26bd2aab) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:04.691419 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db23f8b-4b73-478f-87cf-a987ebe2aa75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:04.692996 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db23f8b-4b73-478f-87cf-a987ebe2aa75 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:04.700418 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 51/95] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:49:04 2022] GET /identity/v3/role_assignments?user.id=9b2edac556674677ba6894dc26bd2aab&scope.project.id=b1e275c904b64687b29e7ca3d28fdda7&role.id=50e04e9d09e045b0bcf8e1782a6e2f23 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:04.733602 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba751d57-bc2f-46c8-9197-441f655ada0d None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:04.767744 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:04.779808 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8e6c98-c213-44cb-9aa1-6a6cd7cf89fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:04.781259 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e6c98-c213-44cb-9aa1-6a6cd7cf89fd None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:04.781744 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e6c98-c213-44cb-9aa1-6a6cd7cf89fd None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:04.782223 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e6c98-c213-44cb-9aa1-6a6cd7cf89fd None admin] PATH_INFO: `/v3/projects/0fa85257194347128519249f6161fd94` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:04.788419 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8e6c98-c213-44cb-9aa1-6a6cd7cf89fd None admin] RBAC: Authorizing `identity:get_project(project_id=0fa85257194347128519249f6161fd94)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:04.790238 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8e6c98-c213-44cb-9aa1-6a6cd7cf89fd None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:04.792648 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 45/96] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:49:04 2022] GET /identity/v3/projects/0fa85257194347128519249f6161fd94 => generated 312 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:04.798336 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db23f8b-4b73-478f-87cf-a987ebe2aa75 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:04.980433 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db23f8b-4b73-478f-87cf-a987ebe2aa75 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:05.032869 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ea03d9-7d9a-4295-8bc1-7a5a8400c4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:05.034377 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea03d9-7d9a-4295-8bc1-7a5a8400c4c2 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:05.034861 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea03d9-7d9a-4295-8bc1-7a5a8400c4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:05.035359 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea03d9-7d9a-4295-8bc1-7a5a8400c4c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:05.036726 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ea03d9-7d9a-4295-8bc1-7a5a8400c4c2 None admin] RBAC: Adding query filter params (role.id=c5431c9deadf47e496264ba5b78eeaf6, scope.project.id=0fa85257194347128519249f6161fd94, user.id=9b2edac556674677ba6894dc26bd2aab) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:05.037479 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ea03d9-7d9a-4295-8bc1-7a5a8400c4c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:05.039260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ea03d9-7d9a-4295-8bc1-7a5a8400c4c2 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:05.052944 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 52/97] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:49:04 2022] GET /identity/v3/role_assignments?user.id=9b2edac556674677ba6894dc26bd2aab&scope.project.id=0fa85257194347128519249f6161fd94&role.id=c5431c9deadf47e496264ba5b78eeaf6 => generated 259 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:06.660565 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0018dd-0b34-41ef-9132-562f37a84858 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:06.662140 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0018dd-0b34-41ef-9132-562f37a84858 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:06.662647 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0018dd-0b34-41ef-9132-562f37a84858 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:06.664277 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 46/98] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:06.669888 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5df0d7-02ae-46df-aeaf-b94f31252afe None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:06.670758 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5df0d7-02ae-46df-aeaf-b94f31252afe None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:06.671253 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5df0d7-02ae-46df-aeaf-b94f31252afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:06.692233 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-7b5df0d7-02ae-46df-aeaf-b94f31252afe None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:06.693973 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-7b5df0d7-02ae-46df-aeaf-b94f31252afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:06.750767 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:06.797666 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 53/99] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:49:06.834192 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0018dd-0b34-41ef-9132-562f37a84858 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:06.968148 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0018dd-0b34-41ef-9132-562f37a84858 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:07.023179 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc157c2-b5c6-46a2-b76a-8a456a28e26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:07.024412 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc157c2-b5c6-46a2-b76a-8a456a28e26d None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:07.024924 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc157c2-b5c6-46a2-b76a-8a456a28e26d None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:07.025416 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc157c2-b5c6-46a2-b76a-8a456a28e26d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:07.048533 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc157c2-b5c6-46a2-b76a-8a456a28e26d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:07.056590 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-5dc157c2-b5c6-46a2-b76a-8a456a28e26d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:49:07.056590 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:07.056590 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:07.056590 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:07.056590 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:07.056590 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:07.056590 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:07.056590 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:07.056590 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:07.056590 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:07.056590 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:07.056590 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:07.056590 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:07.056590 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:07.056590 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:07.056590 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:07.057550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:07.057550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:07.057550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:07.057550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:07.057550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:07.057550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:07.057550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:07.057550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:07.057550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:07.057550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:07.057550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:07.057550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:07.057550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:07.057550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:07.057550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:07.057550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:07.058335 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:07.058335 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:07.058335 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:07.058335 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:07.058335 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:07.058335 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:07.058335 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:07.058335 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:07.058335 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:07.058335 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:07.058335 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:49:07.058335 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:49:07.064735 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 47/100] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jun 7 21:49:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:49:07.071074 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5df0d7-02ae-46df-aeaf-b94f31252afe None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:07.202468 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5df0d7-02ae-46df-aeaf-b94f31252afe None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:07.245348 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9044be-4879-4de8-b4ce-c05826c51082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:07.246338 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9044be-4879-4de8-b4ce-c05826c51082 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:07.246851 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9044be-4879-4de8-b4ce-c05826c51082 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:07.247360 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9044be-4879-4de8-b4ce-c05826c51082 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:07.248332 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9044be-4879-4de8-b4ce-c05826c51082 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:07.248844 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9044be-4879-4de8-b4ce-c05826c51082 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:07.250455 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9044be-4879-4de8-b4ce-c05826c51082 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:07.259181 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2989a-06f6-4dd7-8c69-9fe9566d61b1 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:07.259511 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2989a-06f6-4dd7-8c69-9fe9566d61b1 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:07.259796 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2989a-06f6-4dd7-8c69-9fe9566d61b1 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:07.261173 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 48/101] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:07.265932 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 54/102] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:49:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:07.267258 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917de877-8f4d-495b-b278-ae0894d8234c None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:07.267608 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917de877-8f4d-495b-b278-ae0894d8234c None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:07.267891 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917de877-8f4d-495b-b278-ae0894d8234c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:07.275818 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9044be-4879-4de8-b4ce-c05826c51082 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:07.302437 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-917de877-8f4d-495b-b278-ae0894d8234c None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:07.305261 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-917de877-8f4d-495b-b278-ae0894d8234c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:07.375234 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:07.383033 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:07.445547 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 49/103] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 21:49:07.447220 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f7df9-b51a-4d0c-b388-ad5e0ca32127 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:07.447537 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f7df9-b51a-4d0c-b388-ad5e0ca32127 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:07.447816 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f7df9-b51a-4d0c-b388-ad5e0ca32127 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:07.450140 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 50/104] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:07.456104 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98df55-3433-48d8-8806-5a79eb075526 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:07.456397 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98df55-3433-48d8-8806-5a79eb075526 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:07.459766 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98df55-3433-48d8-8806-5a79eb075526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:07.488699 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-9d98df55-3433-48d8-8806-5a79eb075526 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:07.507274 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9044be-4879-4de8-b4ce-c05826c51082 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:07.509795 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-9d98df55-3433-48d8-8806-5a79eb075526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:07.569621 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb67c6d-77f8-4a12-a57f-18cb84b46a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:07.571930 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb67c6d-77f8-4a12-a57f-18cb84b46a96 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:07.572218 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb67c6d-77f8-4a12-a57f-18cb84b46a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:07.572496 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb67c6d-77f8-4a12-a57f-18cb84b46a96 None admin] PATH_INFO: `/v3/users/c9d023b1a9bd4c969261ea4adeb6b556` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:07.574987 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb67c6d-77f8-4a12-a57f-18cb84b46a96 None admin] RBAC: Authorizing `identity:get_user(user_id=c9d023b1a9bd4c969261ea4adeb6b556)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:07.576476 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb67c6d-77f8-4a12-a57f-18cb84b46a96 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:07.579232 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 55/105] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:49:07 2022] GET /identity/v3/users/c9d023b1a9bd4c969261ea4adeb6b556 => generated 247 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:49:07.580036 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb67c6d-77f8-4a12-a57f-18cb84b46a96 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:07.637275 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 51/106] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:49:07.638144 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d98df55-3433-48d8-8806-5a79eb075526 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:07.772435 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb67c6d-77f8-4a12-a57f-18cb84b46a96 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:07.791368 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:07.823696 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:07.827639 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d98df55-3433-48d8-8806-5a79eb075526 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:07.839727 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db8214d-7c8b-4bcb-8e03-b30a3a0daf21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:07.840544 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8214d-7c8b-4bcb-8e03-b30a3a0daf21 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:07.840878 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8214d-7c8b-4bcb-8e03-b30a3a0daf21 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:07.841159 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8214d-7c8b-4bcb-8e03-b30a3a0daf21 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:07.857822 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db8214d-7c8b-4bcb-8e03-b30a3a0daf21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:07.867147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-6db8214d-7c8b-4bcb-8e03-b30a3a0daf21 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 21:49:07.867147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:07.867147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:07.867147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:07.867147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:07.867147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:07.867147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:07.867147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:07.867147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:07.867147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:07.867147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:07.867147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:07.867147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:07.867147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:07.867147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:07.867147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:07.868076 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:07.868076 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:07.868076 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:07.868076 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:07.868076 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:07.868076 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:07.868076 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:07.868076 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:07.868076 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:07.868076 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:07.868076 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:07.868076 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:07.868076 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:07.868076 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:07.868076 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:07.868076 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:07.869096 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:07.869096 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:07.869096 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:07.869096 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:07.869096 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:07.869096 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:07.869096 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:07.869096 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:07.869096 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:07.869096 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:07.869096 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 21:49:07.869096 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:49:07.869096 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 56/107] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:49:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:49:07.869896 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db8214d-7c8b-4bcb-8e03-b30a3a0daf21 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:07.896555 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a12d98-6cbb-4ef2-8a2a-ac02e9e70682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:07.897429 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a12d98-6cbb-4ef2-8a2a-ac02e9e70682 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:07.897712 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a12d98-6cbb-4ef2-8a2a-ac02e9e70682 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:07.897987 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a12d98-6cbb-4ef2-8a2a-ac02e9e70682 None admin] PATH_INFO: `/v3/projects/b1e275c904b64687b29e7ca3d28fdda7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:07.899188 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a12d98-6cbb-4ef2-8a2a-ac02e9e70682 None admin] RBAC: Authorizing `identity:get_project(project_id=b1e275c904b64687b29e7ca3d28fdda7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:07.900740 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a12d98-6cbb-4ef2-8a2a-ac02e9e70682 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:07.902803 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 52/108] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:49:07 2022] GET /identity/v3/projects/b1e275c904b64687b29e7ca3d28fdda7 => generated 298 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:07.904095 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6f70e-2c01-4675-8b4d-6090d1fa7d70 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:07.904379 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6f70e-2c01-4675-8b4d-6090d1fa7d70 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:07.904655 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6f70e-2c01-4675-8b4d-6090d1fa7d70 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:07.905892 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 53/109] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:07.906590 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c6f70e-2c01-4675-8b4d-6090d1fa7d70 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:07.986741 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db8214d-7c8b-4bcb-8e03-b30a3a0daf21 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:08.015815 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c6f70e-2c01-4675-8b4d-6090d1fa7d70 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:08.027983 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d52d05-88b2-4f6e-a397-7454de19ce75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:08.028801 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d52d05-88b2-4f6e-a397-7454de19ce75 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:08.029095 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d52d05-88b2-4f6e-a397-7454de19ce75 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:08.029382 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d52d05-88b2-4f6e-a397-7454de19ce75 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:08.041212 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d52d05-88b2-4f6e-a397-7454de19ce75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:08.049729 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-95d52d05-88b2-4f6e-a397-7454de19ce75 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 21:49:08.049729 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:08.049729 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:08.049729 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:08.049729 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:08.049729 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:08.049729 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:08.049729 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:08.049729 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:08.049729 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:08.049729 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:08.049729 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:08.049729 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:08.049729 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:08.049729 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:08.049729 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:08.050628 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:08.050628 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:08.050628 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:08.050628 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:08.050628 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:08.050628 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:08.050628 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:08.050628 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:08.050628 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:08.050628 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:08.050628 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:08.050628 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:08.050628 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:08.050628 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:08.050628 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:08.050628 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:08.052127 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:08.052127 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:08.052127 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:08.052127 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:08.052127 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:08.052127 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:08.052127 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:08.052127 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:08.052127 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:08.052127 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:08.052127 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 21:49:08.052127 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:49:08.056380 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 57/110] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:49:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:49:08.058151 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716be0dc-4676-4449-a19c-177657631472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:08.058423 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d52d05-88b2-4f6e-a397-7454de19ce75 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:08.058833 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716be0dc-4676-4449-a19c-177657631472 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:08.059123 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716be0dc-4676-4449-a19c-177657631472 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:08.059447 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716be0dc-4676-4449-a19c-177657631472 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:08.060190 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716be0dc-4676-4449-a19c-177657631472 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:08.060602 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716be0dc-4676-4449-a19c-177657631472 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:08.062099 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716be0dc-4676-4449-a19c-177657631472 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:08.072590 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 54/111] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jun 7 21:49:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:08.073945 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b090b3-6f59-4f28-a0aa-cb941af6c1d9 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:08.074235 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b090b3-6f59-4f28-a0aa-cb941af6c1d9 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:08.074514 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b090b3-6f59-4f28-a0aa-cb941af6c1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:08.092058 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-12b090b3-6f59-4f28-a0aa-cb941af6c1d9 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:08.093236 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-12b090b3-6f59-4f28-a0aa-cb941af6c1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:08.093816 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:08.122343 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:08.177001 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 55/112] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:49:08.177218 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d52d05-88b2-4f6e-a397-7454de19ce75 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:08.177313 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475107b-2c67-459c-8e0e-9204681658d0 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:08.177313 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475107b-2c67-459c-8e0e-9204681658d0 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:08.177313 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475107b-2c67-459c-8e0e-9204681658d0 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:08.177313 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 56/113] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:08.177313 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5475107b-2c67-459c-8e0e-9204681658d0 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:08.216072 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8134e7-cf54-4bd6-be0c-88949232cf62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:08.217131 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8134e7-cf54-4bd6-be0c-88949232cf62 None admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:08.217419 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8134e7-cf54-4bd6-be0c-88949232cf62 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:08.217811 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8134e7-cf54-4bd6-be0c-88949232cf62 None admin] PATH_INFO: `/v3/projects/b1e275c904b64687b29e7ca3d28fdda7/users/c9d023b1a9bd4c969261ea4adeb6b556/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:08.219863 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8134e7-cf54-4bd6-be0c-88949232cf62 None admin] RBAC: Authorizing `identity:create_grant(project_id=b1e275c904b64687b29e7ca3d28fdda7, user_id=c9d023b1a9bd4c969261ea4adeb6b556, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:08.221706 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8134e7-cf54-4bd6-be0c-88949232cf62 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:08.235605 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 58/114] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jun 7 21:49:08 2022] PUT /identity/v3/projects/b1e275c904b64687b29e7ca3d28fdda7/users/c9d023b1a9bd4c969261ea4adeb6b556/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 21:49:08.236687 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8134e7-cf54-4bd6-be0c-88949232cf62 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:08.299489 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5475107b-2c67-459c-8e0e-9204681658d0 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:08.349929 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b21ddc-e4ba-4889-9c7b-3e220691cae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:08.351116 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b21ddc-e4ba-4889-9c7b-3e220691cae9 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:08.351675 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b21ddc-e4ba-4889-9c7b-3e220691cae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:08.352196 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b21ddc-e4ba-4889-9c7b-3e220691cae9 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:08.353185 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b21ddc-e4ba-4889-9c7b-3e220691cae9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:08.353822 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b21ddc-e4ba-4889-9c7b-3e220691cae9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:08.355596 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b21ddc-e4ba-4889-9c7b-3e220691cae9 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:08.366079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:08.367974 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 57/115] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jun 7 21:49:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:08.369837 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5753f23-2a2a-4a17-b70f-36e36542b9a2 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:08.370572 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5753f23-2a2a-4a17-b70f-36e36542b9a2 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:08.371066 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5753f23-2a2a-4a17-b70f-36e36542b9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:08.391100 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8134e7-cf54-4bd6-be0c-88949232cf62 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:08.392459 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-c5753f23-2a2a-4a17-b70f-36e36542b9a2 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:08.393890 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-c5753f23-2a2a-4a17-b70f-36e36542b9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:08.415042 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:08.437840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcd7169-7120-44e9-881b-4f2e4b2708a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:08.438750 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd7169-7120-44e9-881b-4f2e4b2708a8 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:08.439174 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd7169-7120-44e9-881b-4f2e4b2708a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:08.439553 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd7169-7120-44e9-881b-4f2e4b2708a8 None admin] PATH_INFO: `/v3/users/8f1f1a21224a4f3093848cab630ffa09` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:08.458194 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcd7169-7120-44e9-881b-4f2e4b2708a8 None admin] RBAC: Authorizing `identity:get_user(user_id=8f1f1a21224a4f3093848cab630ffa09)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:08.461007 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcd7169-7120-44e9-881b-4f2e4b2708a8 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:08.463199 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 59/116] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:49:08 2022] GET /identity/v3/users/8f1f1a21224a4f3093848cab630ffa09 => generated 289 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:08.463952 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcd7169-7120-44e9-881b-4f2e4b2708a8 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:08.486177 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 58/117] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:49:08.487368 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5753f23-2a2a-4a17-b70f-36e36542b9a2 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:08.589784 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcd7169-7120-44e9-881b-4f2e4b2708a8 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:08.607129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5753f23-2a2a-4a17-b70f-36e36542b9a2 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:08.626693 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546b35f-665f-42d7-b8c6-3dbc47ad02e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:08.627953 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546b35f-665f-42d7-b8c6-3dbc47ad02e8 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:08.628445 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546b35f-665f-42d7-b8c6-3dbc47ad02e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:08.628915 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546b35f-665f-42d7-b8c6-3dbc47ad02e8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:08.631014 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7546b35f-665f-42d7-b8c6-3dbc47ad02e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:08.632706 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7546b35f-665f-42d7-b8c6-3dbc47ad02e8 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:08.634911 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 60/118] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:49:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:08.635903 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546b35f-665f-42d7-b8c6-3dbc47ad02e8 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:08.657906 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:08.660185 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbad8c7-28cc-4811-b70a-88138e15b05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:08.662011 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbad8c7-28cc-4811-b70a-88138e15b05b None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:08.662383 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbad8c7-28cc-4811-b70a-88138e15b05b None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:08.663243 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbad8c7-28cc-4811-b70a-88138e15b05b None admin] PATH_INFO: `/v3/users/9b2edac556674677ba6894dc26bd2aab` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:08.664808 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbad8c7-28cc-4811-b70a-88138e15b05b None admin] RBAC: Authorizing `identity:get_user(user_id=9b2edac556674677ba6894dc26bd2aab)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:08.666762 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbad8c7-28cc-4811-b70a-88138e15b05b None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:08.673312 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 59/119] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:49:08 2022] GET /identity/v3/users/9b2edac556674677ba6894dc26bd2aab => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:08.673312 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbad8c7-28cc-4811-b70a-88138e15b05b None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:08.697105 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:08.746153 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546b35f-665f-42d7-b8c6-3dbc47ad02e8 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:08.788525 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbad8c7-28cc-4811-b70a-88138e15b05b None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:08.789879 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5f9ff-4356-46b8-9ec2-36fd9def59e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:08.790600 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5f9ff-4356-46b8-9ec2-36fd9def59e3 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:08.790953 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5f9ff-4356-46b8-9ec2-36fd9def59e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:08.791400 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5f9ff-4356-46b8-9ec2-36fd9def59e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:08.827406 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32486da1-6ea6-4bc1-8b67-e32e716b0d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:08.828799 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32486da1-6ea6-4bc1-8b67-e32e716b0d99 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:08.829226 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32486da1-6ea6-4bc1-8b67-e32e716b0d99 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:08.829651 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32486da1-6ea6-4bc1-8b67-e32e716b0d99 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:08.840633 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32486da1-6ea6-4bc1-8b67-e32e716b0d99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:08.849682 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-32486da1-6ea6-4bc1-8b67-e32e716b0d99 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 21:49:08.849682 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:08.849682 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:08.849682 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:08.849682 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:08.849682 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:08.849682 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:08.849682 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:08.849682 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:08.849682 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:08.849682 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:08.849682 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:08.849682 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:08.849682 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:08.849682 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:08.849682 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:08.850648 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:08.850648 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:08.850648 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:08.850648 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:08.850648 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:08.850648 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:08.850648 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:08.850648 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:08.850648 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:08.850648 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:08.850648 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:08.850648 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:08.850648 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:08.850648 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:08.850648 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:08.850648 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:08.851567 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:08.851567 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:08.851567 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:08.851567 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:08.851567 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:08.851567 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:08.851567 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:08.851567 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:08.851567 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:08.851567 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:08.851567 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 21:49:08.851567 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:49:08.852661 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 60/120] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Jun 7 21:49:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:49:08.853450 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32486da1-6ea6-4bc1-8b67-e32e716b0d99 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:08.869473 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b5f9ff-4356-46b8-9ec2-36fd9def59e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:08.871203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b5f9ff-4356-46b8-9ec2-36fd9def59e3 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:08.889104 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:08.937987 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:08.958455 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32486da1-6ea6-4bc1-8b67-e32e716b0d99 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:08.980164 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 61/121] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:49:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 21:49:08.982139 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d91be0-26e3-41d8-956f-6ed5b201677a None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:08.982597 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d91be0-26e3-41d8-956f-6ed5b201677a None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:08.983440 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d91be0-26e3-41d8-956f-6ed5b201677a None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:08.985312 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 62/122] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:08.986192 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d91be0-26e3-41d8-956f-6ed5b201677a None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:08.997667 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9cfc7b-2059-4718-a668-ab046c5860da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:08.998344 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9cfc7b-2059-4718-a668-ab046c5860da None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:08.999065 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9cfc7b-2059-4718-a668-ab046c5860da None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:08.999399 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9cfc7b-2059-4718-a668-ab046c5860da None admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:09.000498 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9cfc7b-2059-4718-a668-ab046c5860da None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:09.002245 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9cfc7b-2059-4718-a668-ab046c5860da None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:09.033417 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 61/123] 10.222.0.13 () {60 vars in 1141 bytes} [Tue Jun 7 21:49:08 2022] POST /identity/v3/projects => generated 302 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 21:49:09.034102 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9cfc7b-2059-4718-a668-ab046c5860da None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:09.117633 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d91be0-26e3-41d8-956f-6ed5b201677a None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:09.162084 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:09.167630 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1705a1-d3c0-487f-8bb7-d8b8d4234a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:09.168869 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1705a1-d3c0-487f-8bb7-d8b8d4234a08 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:09.169212 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1705a1-d3c0-487f-8bb7-d8b8d4234a08 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:09.169492 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1705a1-d3c0-487f-8bb7-d8b8d4234a08 None admin] PATH_INFO: `/v3/projects/b1e275c904b64687b29e7ca3d28fdda7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:09.171143 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1705a1-d3c0-487f-8bb7-d8b8d4234a08 None admin] RBAC: Authorizing `identity:get_project(project_id=b1e275c904b64687b29e7ca3d28fdda7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:09.173519 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1705a1-d3c0-487f-8bb7-d8b8d4234a08 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:09.175055 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9cfc7b-2059-4718-a668-ab046c5860da None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:09.179365 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 63/124] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:49:08 2022] GET /identity/v3/projects/b1e275c904b64687b29e7ca3d28fdda7 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:09.179365 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1705a1-d3c0-487f-8bb7-d8b8d4234a08 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:09.224192 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:09.234833 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7d0520-75b5-45ee-b84c-5ec692a3fdc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:09.236228 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d0520-75b5-45ee-b84c-5ec692a3fdc1 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:09.237287 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d0520-75b5-45ee-b84c-5ec692a3fdc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:09.237813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d0520-75b5-45ee-b84c-5ec692a3fdc1 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:09.239241 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7d0520-75b5-45ee-b84c-5ec692a3fdc1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:09.239787 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7d0520-75b5-45ee-b84c-5ec692a3fdc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:09.241602 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7d0520-75b5-45ee-b84c-5ec692a3fdc1 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:09.253117 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 62/125] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:49:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:09.254966 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404ecc8-ba87-4a9f-83f1-a9e3d01ffef3 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:09.255264 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404ecc8-ba87-4a9f-83f1-a9e3d01ffef3 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:09.255570 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404ecc8-ba87-4a9f-83f1-a9e3d01ffef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:09.273408 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-5404ecc8-ba87-4a9f-83f1-a9e3d01ffef3 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:09.274606 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-5404ecc8-ba87-4a9f-83f1-a9e3d01ffef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:09.308463 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1705a1-d3c0-487f-8bb7-d8b8d4234a08 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:09.354015 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb8775-2223-44cc-9ded-def8749b2d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:09.355025 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb8775-2223-44cc-9ded-def8749b2d52 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:09.355310 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb8775-2223-44cc-9ded-def8749b2d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:09.355611 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb8775-2223-44cc-9ded-def8749b2d52 None admin] PATH_INFO: `/v3/projects/b1e275c904b64687b29e7ca3d28fdda7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:09.356968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cb8775-2223-44cc-9ded-def8749b2d52 None admin] RBAC: Authorizing `identity:get_project(project_id=b1e275c904b64687b29e7ca3d28fdda7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:09.359058 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cb8775-2223-44cc-9ded-def8749b2d52 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:09.361769 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 64/126] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:49:09 2022] GET /identity/v3/projects/b1e275c904b64687b29e7ca3d28fdda7 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:09.362585 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb8775-2223-44cc-9ded-def8749b2d52 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:09.370035 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 63/127] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:49:09.375761 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5404ecc8-ba87-4a9f-83f1-a9e3d01ffef3 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:09.453424 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:09.507080 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5404ecc8-ba87-4a9f-83f1-a9e3d01ffef3 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:09.521127 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb8775-2223-44cc-9ded-def8749b2d52 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:09.534645 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:09.547431 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be9b112-aad3-4843-a174-188f02d42fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:09.548367 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be9b112-aad3-4843-a174-188f02d42fd4 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:09.548683 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be9b112-aad3-4843-a174-188f02d42fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:09.548998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be9b112-aad3-4843-a174-188f02d42fd4 None admin] PATH_INFO: `/v3/users/9b2edac556674677ba6894dc26bd2aab` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:09.550494 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be9b112-aad3-4843-a174-188f02d42fd4 None admin] RBAC: Authorizing `identity:get_user(user_id=9b2edac556674677ba6894dc26bd2aab)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:09.552624 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be9b112-aad3-4843-a174-188f02d42fd4 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:09.554718 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 64/128] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:49:09 2022] GET /identity/v3/users/9b2edac556674677ba6894dc26bd2aab => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:09.555522 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be9b112-aad3-4843-a174-188f02d42fd4 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:09.568049 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991bf72-e689-42f7-98fe-d77b1ec82f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:09.569934 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991bf72-e689-42f7-98fe-d77b1ec82f21 None admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:09.570437 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991bf72-e689-42f7-98fe-d77b1ec82f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:09.570927 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991bf72-e689-42f7-98fe-d77b1ec82f21 None admin] PATH_INFO: `/v3/projects/b1e275c904b64687b29e7ca3d28fdda7/users/9b2edac556674677ba6894dc26bd2aab/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:09.574113 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3991bf72-e689-42f7-98fe-d77b1ec82f21 None admin] RBAC: Authorizing `identity:create_grant(project_id=b1e275c904b64687b29e7ca3d28fdda7, user_id=9b2edac556674677ba6894dc26bd2aab, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:09.576416 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3991bf72-e689-42f7-98fe-d77b1ec82f21 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:09.593947 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 65/129] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jun 7 21:49:09 2022] PUT /identity/v3/projects/b1e275c904b64687b29e7ca3d28fdda7/users/9b2edac556674677ba6894dc26bd2aab/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 21:49:09.595412 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991bf72-e689-42f7-98fe-d77b1ec82f21 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:09.709177 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991bf72-e689-42f7-98fe-d77b1ec82f21 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:09.722954 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:09.751729 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eb3c3e-6af2-4067-a6fe-be5553e598d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:09.752532 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb3c3e-6af2-4067-a6fe-be5553e598d4 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:09.752670 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be9b112-aad3-4843-a174-188f02d42fd4 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:09.752843 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb3c3e-6af2-4067-a6fe-be5553e598d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:09.753084 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb3c3e-6af2-4067-a6fe-be5553e598d4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:09.766805 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eb3c3e-6af2-4067-a6fe-be5553e598d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:09.777481 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-b3eb3c3e-6af2-4067-a6fe-be5553e598d4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 21:49:09.777481 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:09.777481 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:09.777481 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:09.777481 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:09.777481 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:09.777481 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:09.777481 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:09.777481 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:09.777481 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:09.777481 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:09.777481 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:09.777481 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:09.777481 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:09.777481 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:09.777481 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:09.779351 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:09.779351 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:09.779351 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:09.779351 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:09.779351 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:09.779351 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:09.779351 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:09.779351 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:09.779351 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:09.779351 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:09.779351 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:09.779351 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:09.779351 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:09.779351 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:09.779351 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:09.779351 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:09.780676 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:09.780676 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:09.780676 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:09.780676 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:09.780676 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:09.780676 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:09.780676 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:09.780676 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:09.780676 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:09.780676 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:09.780676 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 21:49:09.780676 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:49:09.780676 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 66/130] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:49:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:49:09.780676 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eb3c3e-6af2-4067-a6fe-be5553e598d4 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:09.894247 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eb3c3e-6af2-4067-a6fe-be5553e598d4 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:09.902731 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f73f076-f540-4042-9b79-4dc78cb835e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:09.903519 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73f076-f540-4042-9b79-4dc78cb835e9 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:09.903809 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73f076-f540-4042-9b79-4dc78cb835e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:09.904091 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73f076-f540-4042-9b79-4dc78cb835e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:09.905014 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f73f076-f540-4042-9b79-4dc78cb835e9 None admin] RBAC: Adding query filter params (role.id=30b0c2ba85ff481eb256122140e4163a, scope.project.id=b1e275c904b64687b29e7ca3d28fdda7, user.id=8f1f1a21224a4f3093848cab630ffa09) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:09.905338 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f73f076-f540-4042-9b79-4dc78cb835e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:09.906819 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f73f076-f540-4042-9b79-4dc78cb835e9 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:09.924087 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 65/131] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:49:09 2022] GET /identity/v3/role_assignments?user.id=8f1f1a21224a4f3093848cab630ffa09&scope.project.id=b1e275c904b64687b29e7ca3d28fdda7&role.id=30b0c2ba85ff481eb256122140e4163a => generated 259 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:09.925033 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f73f076-f540-4042-9b79-4dc78cb835e9 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:09.935378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e0d4be-0c35-43a2-9fe2-88bf1b9e03b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:09.936268 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e0d4be-0c35-43a2-9fe2-88bf1b9e03b5 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:09.936552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e0d4be-0c35-43a2-9fe2-88bf1b9e03b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:09.937443 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e0d4be-0c35-43a2-9fe2-88bf1b9e03b5 None admin] PATH_INFO: `/v3/projects/b1e275c904b64687b29e7ca3d28fdda7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:09.938712 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e0d4be-0c35-43a2-9fe2-88bf1b9e03b5 None admin] RBAC: Authorizing `identity:get_project(project_id=b1e275c904b64687b29e7ca3d28fdda7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:09.940305 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e0d4be-0c35-43a2-9fe2-88bf1b9e03b5 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:09.943041 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 67/132] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:49:09 2022] GET /identity/v3/projects/b1e275c904b64687b29e7ca3d28fdda7 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:09.949662 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e0d4be-0c35-43a2-9fe2-88bf1b9e03b5 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:10.008752 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:10.011842 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:10.075823 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f73f076-f540-4042-9b79-4dc78cb835e9 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:10.093872 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e0d4be-0c35-43a2-9fe2-88bf1b9e03b5 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:10.152265 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad9105e-7d73-45e5-a27b-dc4de734cc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:10.153190 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad9105e-7d73-45e5-a27b-dc4de734cc1d None admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:10.153476 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad9105e-7d73-45e5-a27b-dc4de734cc1d None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:10.153754 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad9105e-7d73-45e5-a27b-dc4de734cc1d None admin] PATH_INFO: `/v3/projects/b1e275c904b64687b29e7ca3d28fdda7/users/9b2edac556674677ba6894dc26bd2aab/roles/50e04e9d09e045b0bcf8e1782a6e2f23` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:10.156780 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad9105e-7d73-45e5-a27b-dc4de734cc1d None admin] RBAC: Authorizing `identity:create_grant(project_id=b1e275c904b64687b29e7ca3d28fdda7, user_id=9b2edac556674677ba6894dc26bd2aab, role_id=50e04e9d09e045b0bcf8e1782a6e2f23)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:10.158274 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad9105e-7d73-45e5-a27b-dc4de734cc1d None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:10.171392 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c5a59c-95f9-4a48-bd7a-0e67cd87a8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:10.172419 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5a59c-95f9-4a48-bd7a-0e67cd87a8d6 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:10.172912 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5a59c-95f9-4a48-bd7a-0e67cd87a8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:10.173210 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5a59c-95f9-4a48-bd7a-0e67cd87a8d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:10.174126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c5a59c-95f9-4a48-bd7a-0e67cd87a8d6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:10.174452 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c5a59c-95f9-4a48-bd7a-0e67cd87a8d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:10.175966 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c5a59c-95f9-4a48-bd7a-0e67cd87a8d6 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:10.183249 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 68/133] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jun 7 21:49:09 2022] PUT /identity/v3/projects/b1e275c904b64687b29e7ca3d28fdda7/users/9b2edac556674677ba6894dc26bd2aab/roles/50e04e9d09e045b0bcf8e1782a6e2f23 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 21:49:10.195379 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 66/134] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jun 7 21:49:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:10.201781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad9105e-7d73-45e5-a27b-dc4de734cc1d None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:10.342277 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:10.347618 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad9105e-7d73-45e5-a27b-dc4de734cc1d None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:10.395769 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02387bd0-c81c-44b9-a9c3-557f35e88987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:10.396674 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02387bd0-c81c-44b9-a9c3-557f35e88987 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:10.396956 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02387bd0-c81c-44b9-a9c3-557f35e88987 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:10.397228 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02387bd0-c81c-44b9-a9c3-557f35e88987 None admin] PATH_INFO: `/v3/users/9b2edac556674677ba6894dc26bd2aab` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:10.404308 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02387bd0-c81c-44b9-a9c3-557f35e88987 None admin] RBAC: Authorizing `identity:get_user(user_id=9b2edac556674677ba6894dc26bd2aab)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:10.405813 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02387bd0-c81c-44b9-a9c3-557f35e88987 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:10.407906 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 69/135] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:49:10 2022] GET /identity/v3/users/9b2edac556674677ba6894dc26bd2aab => generated 275 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:10.414341 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c5a59c-95f9-4a48-bd7a-0e67cd87a8d6 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:10.573004 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c5a59c-95f9-4a48-bd7a-0e67cd87a8d6 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:10.593522 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:10.618927 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40275d09-8e87-4289-8777-f779065510c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:10.619845 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40275d09-8e87-4289-8777-f779065510c7 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:10.620132 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40275d09-8e87-4289-8777-f779065510c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:10.620411 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40275d09-8e87-4289-8777-f779065510c7 None admin] PATH_INFO: `/v3/projects/0fa85257194347128519249f6161fd94` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:10.621673 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40275d09-8e87-4289-8777-f779065510c7 None admin] RBAC: Authorizing `identity:get_project(project_id=0fa85257194347128519249f6161fd94)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:10.623134 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40275d09-8e87-4289-8777-f779065510c7 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:10.625042 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 67/136] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:49:10 2022] GET /identity/v3/projects/0fa85257194347128519249f6161fd94 => generated 312 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:10.631238 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02387bd0-c81c-44b9-a9c3-557f35e88987 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:10.743620 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02387bd0-c81c-44b9-a9c3-557f35e88987 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:10.789880 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268e5eeb-6126-4838-b43e-7220be1d93ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:10.791145 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e5eeb-6126-4838-b43e-7220be1d93ad None admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:10.791692 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e5eeb-6126-4838-b43e-7220be1d93ad None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:10.792173 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e5eeb-6126-4838-b43e-7220be1d93ad None admin] PATH_INFO: `/v3/projects/0fa85257194347128519249f6161fd94/users/9b2edac556674677ba6894dc26bd2aab/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:10.797711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268e5eeb-6126-4838-b43e-7220be1d93ad None admin] RBAC: Authorizing `identity:create_grant(project_id=0fa85257194347128519249f6161fd94, user_id=9b2edac556674677ba6894dc26bd2aab, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:10.799534 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268e5eeb-6126-4838-b43e-7220be1d93ad None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:10.825115 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 70/137] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jun 7 21:49:10 2022] PUT /identity/v3/projects/0fa85257194347128519249f6161fd94/users/9b2edac556674677ba6894dc26bd2aab/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 21:49:10.887057 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087267d-aa35-4a67-93a8-8d394fa0e502 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:10.887268 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087267d-aa35-4a67-93a8-8d394fa0e502 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:10.887567 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087267d-aa35-4a67-93a8-8d394fa0e502 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:10.889136 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 68/138] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:10.895551 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf33e07a-c9dc-42ee-9ac2-7308e3e9bd79 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:10.896536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf33e07a-c9dc-42ee-9ac2-7308e3e9bd79 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:10.896963 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf33e07a-c9dc-42ee-9ac2-7308e3e9bd79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:10.915789 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-bf33e07a-c9dc-42ee-9ac2-7308e3e9bd79 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:10.917249 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-bf33e07a-c9dc-42ee-9ac2-7308e3e9bd79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:10.942124 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:10.993471 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 71/139] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:49:11.017353 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0087267d-aa35-4a67-93a8-8d394fa0e502 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:11.153581 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0087267d-aa35-4a67-93a8-8d394fa0e502 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:11.206381 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dff4e5-09fc-47aa-b09d-715fdab30935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:11.207186 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dff4e5-09fc-47aa-b09d-715fdab30935 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:11.208159 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dff4e5-09fc-47aa-b09d-715fdab30935 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:11.208454 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dff4e5-09fc-47aa-b09d-715fdab30935 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:11.227295 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dff4e5-09fc-47aa-b09d-715fdab30935 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:11.235129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-f6dff4e5-09fc-47aa-b09d-715fdab30935 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:49:11.235129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:11.235129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:11.235129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:11.235129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:11.235129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:11.235129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:11.235129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:11.235129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:11.235129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:11.235129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:11.235129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:11.235129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:11.235129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:11.235129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:11.235129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:11.236004 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:11.236004 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:11.236004 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:11.236004 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:11.236004 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:11.236004 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:11.236004 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:11.236004 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:11.236004 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:11.236004 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:11.236004 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:11.236004 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:11.236004 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:11.236004 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:11.236004 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:11.236004 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:11.236771 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:11.236771 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:11.236771 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:11.236771 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:11.236771 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:11.236771 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:11.236771 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:11.236771 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:11.236771 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:11.236771 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:11.236771 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:49:11.236771 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:49:11.236771 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 69/140] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jun 7 21:49:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:49:11.242235 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf33e07a-c9dc-42ee-9ac2-7308e3e9bd79 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:11.366660 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf33e07a-c9dc-42ee-9ac2-7308e3e9bd79 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:11.402643 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7297fb73-9bbd-4f74-8d7d-e164b356014b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:11.403341 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297fb73-9bbd-4f74-8d7d-e164b356014b None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:11.407001 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297fb73-9bbd-4f74-8d7d-e164b356014b None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:11.407239 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297fb73-9bbd-4f74-8d7d-e164b356014b None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:11.408111 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7297fb73-9bbd-4f74-8d7d-e164b356014b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:11.408429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7297fb73-9bbd-4f74-8d7d-e164b356014b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:11.410217 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7297fb73-9bbd-4f74-8d7d-e164b356014b None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:11.420053 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 72/141] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:49:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:11.425587 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dff4e5-09fc-47aa-b09d-715fdab30935 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:11.511017 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:11.579820 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dff4e5-09fc-47aa-b09d-715fdab30935 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:11.627115 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef60ea81-579e-49c5-8417-66c1e746e43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:11.628036 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef60ea81-579e-49c5-8417-66c1e746e43d None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:11.628321 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef60ea81-579e-49c5-8417-66c1e746e43d None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:11.628599 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef60ea81-579e-49c5-8417-66c1e746e43d None admin] PATH_INFO: `/v3/users/c9d023b1a9bd4c969261ea4adeb6b556` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:11.630223 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef60ea81-579e-49c5-8417-66c1e746e43d None admin] RBAC: Authorizing `identity:get_user(user_id=c9d023b1a9bd4c969261ea4adeb6b556)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:11.631707 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef60ea81-579e-49c5-8417-66c1e746e43d None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:11.633523 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 70/142] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:49:11 2022] GET /identity/v3/users/c9d023b1a9bd4c969261ea4adeb6b556 => generated 247 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:11.638880 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7297fb73-9bbd-4f74-8d7d-e164b356014b None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:11.662844 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:11.749847 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7297fb73-9bbd-4f74-8d7d-e164b356014b None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:11.793487 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9071c2b4-6840-4ad9-9fa5-7acbac4bbf8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:11.794143 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9071c2b4-6840-4ad9-9fa5-7acbac4bbf8f None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:11.794433 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9071c2b4-6840-4ad9-9fa5-7acbac4bbf8f None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:11.794755 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9071c2b4-6840-4ad9-9fa5-7acbac4bbf8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:11.865667 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9071c2b4-6840-4ad9-9fa5-7acbac4bbf8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:11.867156 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9071c2b4-6840-4ad9-9fa5-7acbac4bbf8f None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:11.889335 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:11.977391 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 73/143] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:49:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 21:49:11.980225 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef60ea81-579e-49c5-8417-66c1e746e43d None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:12.093761 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef60ea81-579e-49c5-8417-66c1e746e43d None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:12.109289 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:12.132288 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937b4035-1aad-4e7f-8af5-efb05f69fff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:12.133170 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b4035-1aad-4e7f-8af5-efb05f69fff2 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:12.133455 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b4035-1aad-4e7f-8af5-efb05f69fff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:12.133734 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b4035-1aad-4e7f-8af5-efb05f69fff2 None admin] PATH_INFO: `/v3/projects/b1e275c904b64687b29e7ca3d28fdda7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:12.134946 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937b4035-1aad-4e7f-8af5-efb05f69fff2 None admin] RBAC: Authorizing `identity:get_project(project_id=b1e275c904b64687b29e7ca3d28fdda7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:12.136444 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937b4035-1aad-4e7f-8af5-efb05f69fff2 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:12.138139 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 71/144] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:49:11 2022] GET /identity/v3/projects/b1e275c904b64687b29e7ca3d28fdda7 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:12.144368 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9071c2b4-6840-4ad9-9fa5-7acbac4bbf8f None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:12.251022 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9071c2b4-6840-4ad9-9fa5-7acbac4bbf8f None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:12.272258 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:12.286641 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff864d2-e5ad-4a2e-9a38-f4e2f5ee5e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:12.287299 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff864d2-e5ad-4a2e-9a38-f4e2f5ee5e2f None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:12.287609 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff864d2-e5ad-4a2e-9a38-f4e2f5ee5e2f None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:12.287884 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff864d2-e5ad-4a2e-9a38-f4e2f5ee5e2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:12.288720 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff864d2-e5ad-4a2e-9a38-f4e2f5ee5e2f None admin] RBAC: Adding query filter params (role.id=6b483758dbd44eb4a78df2ce4603d601, scope.project.id=b1e275c904b64687b29e7ca3d28fdda7, user.id=c9d023b1a9bd4c969261ea4adeb6b556) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:12.289026 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff864d2-e5ad-4a2e-9a38-f4e2f5ee5e2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:12.290480 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff864d2-e5ad-4a2e-9a38-f4e2f5ee5e2f None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:12.296754 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 74/145] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:49:12 2022] GET /identity/v3/role_assignments?user.id=c9d023b1a9bd4c969261ea4adeb6b556&scope.project.id=b1e275c904b64687b29e7ca3d28fdda7&role.id=6b483758dbd44eb4a78df2ce4603d601 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:12.341635 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655eeb5-12de-4740-b21d-f58d24f878f7 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:12.341976 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655eeb5-12de-4740-b21d-f58d24f878f7 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:12.342195 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655eeb5-12de-4740-b21d-f58d24f878f7 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:12.343944 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 72/146] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:12.349885 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296bca00-97cb-43e6-a59d-18238665bebc None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:12.350167 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296bca00-97cb-43e6-a59d-18238665bebc None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:12.350440 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296bca00-97cb-43e6-a59d-18238665bebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:12.370621 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-296bca00-97cb-43e6-a59d-18238665bebc None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:12.371832 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-296bca00-97cb-43e6-a59d-18238665bebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:12.448040 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 75/147] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:49:12.495231 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8655eeb5-12de-4740-b21d-f58d24f878f7 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:12.628085 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8655eeb5-12de-4740-b21d-f58d24f878f7 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:12.671519 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91ca241-e940-492a-9f29-5c12a2ae745a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:12.672335 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ca241-e940-492a-9f29-5c12a2ae745a None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:12.672612 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ca241-e940-492a-9f29-5c12a2ae745a None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:12.672883 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ca241-e940-492a-9f29-5c12a2ae745a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:12.674052 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91ca241-e940-492a-9f29-5c12a2ae745a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:12.675577 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91ca241-e940-492a-9f29-5c12a2ae745a None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:12.677361 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 73/148] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:49:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:12.683048 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296bca00-97cb-43e6-a59d-18238665bebc None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:12.794344 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296bca00-97cb-43e6-a59d-18238665bebc None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:12.809155 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:12.830373 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fabc80f-1663-4f8b-970f-03b12610780b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:12.831045 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabc80f-1663-4f8b-970f-03b12610780b None admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:12.831341 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabc80f-1663-4f8b-970f-03b12610780b None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:12.831629 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabc80f-1663-4f8b-970f-03b12610780b None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:12.832381 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fabc80f-1663-4f8b-970f-03b12610780b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:12.833994 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fabc80f-1663-4f8b-970f-03b12610780b None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:12.857182 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 76/149] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Jun 7 21:49:12 2022] POST /identity/v3/users => generated 283 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 21:49:13.431958 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904236a-deb8-4b2b-b94a-c23315fb4910 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:13.432501 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904236a-deb8-4b2b-b94a-c23315fb4910 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:13.432501 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904236a-deb8-4b2b-b94a-c23315fb4910 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:13.434136 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 74/150] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:13.440272 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83db01-609b-4cfb-a18b-734c6c81351b None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:13.440766 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83db01-609b-4cfb-a18b-734c6c81351b None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:13.440837 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83db01-609b-4cfb-a18b-734c6c81351b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:13.460669 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-3f83db01-609b-4cfb-a18b-734c6c81351b None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:13.462314 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-3f83db01-609b-4cfb-a18b-734c6c81351b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:13.544314 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 77/151] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:49:13.572538 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e904236a-deb8-4b2b-b94a-c23315fb4910 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:13.608280 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25748a8-b9c9-436f-bb62-fda88e119324 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:13.608910 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25748a8-b9c9-436f-bb62-fda88e119324 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:13.609376 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25748a8-b9c9-436f-bb62-fda88e119324 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:13.610928 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 78/152] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 21:49:13.616904 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf761c7a-6b52-4eae-a839-6d531da7899d None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:13.617461 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf761c7a-6b52-4eae-a839-6d531da7899d None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:13.617951 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf761c7a-6b52-4eae-a839-6d531da7899d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:13.640938 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-cf761c7a-6b52-4eae-a839-6d531da7899d None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:13.642442 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-cf761c7a-6b52-4eae-a839-6d531da7899d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:13.646077 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:13.695103 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e904236a-deb8-4b2b-b94a-c23315fb4910 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:13.726436 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 79/153] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 21:49:13.741148 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed0a98f-61a9-4a06-a0fe-d7dd40ee5df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:13.741947 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed0a98f-61a9-4a06-a0fe-d7dd40ee5df3 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:13.742225 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed0a98f-61a9-4a06-a0fe-d7dd40ee5df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:13.742495 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed0a98f-61a9-4a06-a0fe-d7dd40ee5df3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:13.759126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed0a98f-61a9-4a06-a0fe-d7dd40ee5df3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:13.771546 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf761c7a-6b52-4eae-a839-6d531da7899d None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:13.773151 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-eed0a98f-61a9-4a06-a0fe-d7dd40ee5df3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 21:49:13.773151 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:13.773151 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:13.773151 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:13.773151 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:13.773151 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:13.773151 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:13.773151 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:13.773151 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:13.773151 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:13.773151 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:13.773151 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:13.773151 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:13.773151 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:13.773151 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:13.773151 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:13.774007 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:13.774007 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:13.774007 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:13.774007 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:13.774007 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:13.774007 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:13.774007 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:13.774007 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:13.774007 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:13.774007 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:13.774007 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:13.774007 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:13.774007 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:13.774007 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:13.774007 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:13.774007 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:13.779509 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:13.779509 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:13.779509 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:13.779509 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:13.779509 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:13.779509 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:13.779509 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:13.779509 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:13.779509 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:13.779509 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:13.779509 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 21:49:13.779509 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:49:13.779509 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 75/154] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:49:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:49:13.784300 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed0a98f-61a9-4a06-a0fe-d7dd40ee5df3 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:13.819027 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:13.907684 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf761c7a-6b52-4eae-a839-6d531da7899d None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:13.940271 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed0a98f-61a9-4a06-a0fe-d7dd40ee5df3 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:13.953529 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:13.953680 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe5ceac-366c-42cc-8212-7ba3b753d21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:13.955058 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5ceac-366c-42cc-8212-7ba3b753d21c None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:13.955605 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5ceac-366c-42cc-8212-7ba3b753d21c None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:13.955950 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5ceac-366c-42cc-8212-7ba3b753d21c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:13.970118 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe5ceac-366c-42cc-8212-7ba3b753d21c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:13.981117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-dfe5ceac-366c-42cc-8212-7ba3b753d21c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 21:49:13.981117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:13.981117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:13.981117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:13.981117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:13.981117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:13.981117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:13.981117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:13.981117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:13.981117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:13.981117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:13.981117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:13.981117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:13.981117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:13.981117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:13.981117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:13.982203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:13.982203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:13.982203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:13.982203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:13.982203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:13.982203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:13.982203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:13.982203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:13.982203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:13.982203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:13.982203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:13.982203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:13.982203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:13.982203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:13.982203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:13.982203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:13.982998 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:13.982998 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:13.982998 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:13.982998 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:13.982998 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:13.982998 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:13.982998 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:13.982998 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:13.982998 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:13.982998 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:13.982998 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 21:49:13.982998 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:49:13.984716 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 80/155] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:49:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:49:13.989920 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d500e-1ed0-4bc7-b311-0608ffc76535 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:13.989920 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d500e-1ed0-4bc7-b311-0608ffc76535 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:13.990125 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bb209f-9886-4e9c-8e7f-67453dea8a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:13.990125 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb209f-9886-4e9c-8e7f-67453dea8a0f None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:13.990125 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb209f-9886-4e9c-8e7f-67453dea8a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:13.990125 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb209f-9886-4e9c-8e7f-67453dea8a0f None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:13.990384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bb209f-9886-4e9c-8e7f-67453dea8a0f None admin] RBAC: Adding query filter params (name=member) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:13.990384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bb209f-9886-4e9c-8e7f-67453dea8a0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:13.991162 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d500e-1ed0-4bc7-b311-0608ffc76535 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:13.992771 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 81/156] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:13.993538 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6d500e-1ed0-4bc7-b311-0608ffc76535 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:13.994510 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bb209f-9886-4e9c-8e7f-67453dea8a0f None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:14.005756 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 76/157] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jun 7 21:49:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:14.006451 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e3710-97e1-48e7-ba7b-7aea7cec5f6f None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:14.006745 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e3710-97e1-48e7-ba7b-7aea7cec5f6f None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:14.007023 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e3710-97e1-48e7-ba7b-7aea7cec5f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:14.028422 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-b89e3710-97e1-48e7-ba7b-7aea7cec5f6f None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:14.029680 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-b89e3710-97e1-48e7-ba7b-7aea7cec5f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:14.121112 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:14.124682 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 77/158] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:49:14.125755 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89e3710-97e1-48e7-ba7b-7aea7cec5f6f None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:14.136022 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6d500e-1ed0-4bc7-b311-0608ffc76535 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:14.182658 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f10c6b9-58c8-4349-8e88-e113f057db15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:14.183595 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f10c6b9-58c8-4349-8e88-e113f057db15 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:14.184067 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f10c6b9-58c8-4349-8e88-e113f057db15 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:14.184480 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f10c6b9-58c8-4349-8e88-e113f057db15 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:14.185562 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f10c6b9-58c8-4349-8e88-e113f057db15 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:14.186028 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f10c6b9-58c8-4349-8e88-e113f057db15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:14.187784 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f10c6b9-58c8-4349-8e88-e113f057db15 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:14.199473 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 82/159] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jun 7 21:49:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:14.200635 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f10c6b9-58c8-4349-8e88-e113f057db15 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:14.268373 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89e3710-97e1-48e7-ba7b-7aea7cec5f6f None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:14.269965 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:14.315917 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eedc51-650e-4ee6-8ea0-9e7897de90a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:14.317352 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eedc51-650e-4ee6-8ea0-9e7897de90a8 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:14.317639 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eedc51-650e-4ee6-8ea0-9e7897de90a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:14.317917 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eedc51-650e-4ee6-8ea0-9e7897de90a8 None admin] PATH_INFO: `/v3/users/9b2edac556674677ba6894dc26bd2aab` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:14.319146 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eedc51-650e-4ee6-8ea0-9e7897de90a8 None admin] RBAC: Authorizing `identity:get_user(user_id=9b2edac556674677ba6894dc26bd2aab)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:14.320640 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eedc51-650e-4ee6-8ea0-9e7897de90a8 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:14.322455 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 78/160] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:49:14 2022] GET /identity/v3/users/9b2edac556674677ba6894dc26bd2aab => generated 275 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:14.327896 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2077b3-7908-4b54-b412-596de87e764f None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:14.328193 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2077b3-7908-4b54-b412-596de87e764f None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:14.328474 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2077b3-7908-4b54-b412-596de87e764f None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:14.330043 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 79/161] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:14.330530 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f10c6b9-58c8-4349-8e88-e113f057db15 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:14.330762 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2077b3-7908-4b54-b412-596de87e764f None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:14.391938 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db4ee5-547a-4b8e-b4bb-6fd20e5c68fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:14.394107 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db4ee5-547a-4b8e-b4bb-6fd20e5c68fb None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:14.394397 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db4ee5-547a-4b8e-b4bb-6fd20e5c68fb None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:14.394741 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db4ee5-547a-4b8e-b4bb-6fd20e5c68fb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:14.415016 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6db4ee5-547a-4b8e-b4bb-6fd20e5c68fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:14.429388 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-e6db4ee5-547a-4b8e-b4bb-6fd20e5c68fb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 21:49:14.429388 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:14.429388 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:14.429388 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:14.429388 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:14.429388 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:14.429388 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:14.429388 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:14.429388 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:14.429388 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:14.429388 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:14.429388 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:14.429388 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:14.429388 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:14.429388 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:14.429388 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:14.430322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:14.430322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:14.430322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:14.430322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:14.430322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:14.430322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:14.430322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:14.430322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:14.430322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:14.430322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:14.430322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:14.430322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:14.430322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:14.430322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:14.430322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:14.430322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:14.431610 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:14.431610 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:14.431610 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:14.431610 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:14.431610 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:14.431610 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:14.431610 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:14.431610 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:14.431610 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:14.431610 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:14.431610 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 21:49:14.431610 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:49:14.431610 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 83/162] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Jun 7 21:49:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:49:14.431610 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db4ee5-547a-4b8e-b4bb-6fd20e5c68fb None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:14.449793 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:14.504036 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2077b3-7908-4b54-b412-596de87e764f None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:14.549984 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db4ee5-547a-4b8e-b4bb-6fd20e5c68fb None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:14.589048 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb3692-c83d-45d9-9d6b-59fa64e41fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:14.589694 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb3692-c83d-45d9-9d6b-59fa64e41fbe None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:14.589976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb3692-c83d-45d9-9d6b-59fa64e41fbe None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:14.590249 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb3692-c83d-45d9-9d6b-59fa64e41fbe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:14.604601 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5732fe3-aca2-47e3-b88e-e796ac909e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:14.605495 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5732fe3-aca2-47e3-b88e-e796ac909e37 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:14.605779 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5732fe3-aca2-47e3-b88e-e796ac909e37 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:14.606054 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5732fe3-aca2-47e3-b88e-e796ac909e37 None admin] PATH_INFO: `/v3/users/8f1f1a21224a4f3093848cab630ffa09` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:14.612050 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5732fe3-aca2-47e3-b88e-e796ac909e37 None admin] RBAC: Authorizing `identity:get_user(user_id=8f1f1a21224a4f3093848cab630ffa09)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:14.614017 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5732fe3-aca2-47e3-b88e-e796ac909e37 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:14.615851 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 80/163] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:49:14 2022] GET /identity/v3/users/8f1f1a21224a4f3093848cab630ffa09 => generated 289 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:14.617087 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7082c690-b0e9-4c42-8969-e83bf8d2fa38 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:14.617381 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7082c690-b0e9-4c42-8969-e83bf8d2fa38 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:14.617664 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7082c690-b0e9-4c42-8969-e83bf8d2fa38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:14.660388 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-7082c690-b0e9-4c42-8969-e83bf8d2fa38 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:14.661803 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-7082c690-b0e9-4c42-8969-e83bf8d2fa38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:14.663812 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eb3692-c83d-45d9-9d6b-59fa64e41fbe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:14.665429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eb3692-c83d-45d9-9d6b-59fa64e41fbe None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:14.681145 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:14.736345 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:14.782182 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 84/164] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:49:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 21:49:14.783037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb3692-c83d-45d9-9d6b-59fa64e41fbe None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:14.831535 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 81/165] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:49:14.836361 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7082c690-b0e9-4c42-8969-e83bf8d2fa38 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:14.904804 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb3692-c83d-45d9-9d6b-59fa64e41fbe None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:14.919786 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:14.946006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f813bc-eaf3-4157-bfd7-c345d130dfca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:14.947703 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f813bc-eaf3-4157-bfd7-c345d130dfca None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:14.947986 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f813bc-eaf3-4157-bfd7-c345d130dfca None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:14.948261 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f813bc-eaf3-4157-bfd7-c345d130dfca None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:14.949017 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f813bc-eaf3-4157-bfd7-c345d130dfca None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:14.949321 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f813bc-eaf3-4157-bfd7-c345d130dfca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:14.950723 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f813bc-eaf3-4157-bfd7-c345d130dfca None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:14.963377 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 85/166] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:49:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:14.964627 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f813bc-eaf3-4157-bfd7-c345d130dfca None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:15.080084 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f813bc-eaf3-4157-bfd7-c345d130dfca None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:15.109272 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7082c690-b0e9-4c42-8969-e83bf8d2fa38 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:15.119309 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee6d117-e68e-4d6f-80cb-13c56391fc4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:15.120693 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6d117-e68e-4d6f-80cb-13c56391fc4f None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:15.120978 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6d117-e68e-4d6f-80cb-13c56391fc4f None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:15.121254 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6d117-e68e-4d6f-80cb-13c56391fc4f None admin] PATH_INFO: `/v3/projects/b1e275c904b64687b29e7ca3d28fdda7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:15.122454 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee6d117-e68e-4d6f-80cb-13c56391fc4f None admin] RBAC: Authorizing `identity:get_project(project_id=b1e275c904b64687b29e7ca3d28fdda7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:15.123925 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee6d117-e68e-4d6f-80cb-13c56391fc4f None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:15.126086 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 86/167] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:49:14 2022] GET /identity/v3/projects/b1e275c904b64687b29e7ca3d28fdda7 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:15.127272 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee6d117-e68e-4d6f-80cb-13c56391fc4f None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:15.181220 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:15.213795 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa582ae9-d5b7-4da7-97c8-e0abc19f46b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:15.214973 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa582ae9-d5b7-4da7-97c8-e0abc19f46b6 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:15.215260 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa582ae9-d5b7-4da7-97c8-e0abc19f46b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:15.215593 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa582ae9-d5b7-4da7-97c8-e0abc19f46b6 None admin] PATH_INFO: `/v3/projects/b1e275c904b64687b29e7ca3d28fdda7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:15.216801 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa582ae9-d5b7-4da7-97c8-e0abc19f46b6 None admin] RBAC: Authorizing `identity:get_project(project_id=b1e275c904b64687b29e7ca3d28fdda7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:15.218276 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa582ae9-d5b7-4da7-97c8-e0abc19f46b6 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:15.220211 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 82/168] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:49:14 2022] GET /identity/v3/projects/b1e275c904b64687b29e7ca3d28fdda7 => generated 298 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:15.221097 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa582ae9-d5b7-4da7-97c8-e0abc19f46b6 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:15.252016 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee6d117-e68e-4d6f-80cb-13c56391fc4f None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:15.275967 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:15.293261 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40997f0b-52a3-44a6-babc-319bc5b5af95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:15.294060 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40997f0b-52a3-44a6-babc-319bc5b5af95 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:15.294342 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40997f0b-52a3-44a6-babc-319bc5b5af95 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:15.294617 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40997f0b-52a3-44a6-babc-319bc5b5af95 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:15.306428 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40997f0b-52a3-44a6-babc-319bc5b5af95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:15.314883 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-40997f0b-52a3-44a6-babc-319bc5b5af95 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 21:49:15.314883 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:15.314883 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:15.314883 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:15.314883 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:15.314883 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:15.314883 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:15.314883 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:15.314883 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:15.314883 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:15.314883 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:15.314883 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:15.314883 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:15.314883 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:15.314883 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:15.314883 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:15.315783 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:15.315783 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:15.315783 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:15.315783 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:15.315783 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:15.315783 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:15.315783 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:15.315783 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:15.315783 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:15.315783 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:15.315783 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:15.315783 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:15.315783 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:15.315783 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:15.315783 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:15.315783 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:15.316633 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:15.316633 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:15.316633 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:15.316633 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:15.316633 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:15.316633 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:15.316633 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:15.316633 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:15.316633 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:15.316633 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:15.316633 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 21:49:15.316633 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:49:15.318556 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 87/169] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:49:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:49:15.318556 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40997f0b-52a3-44a6-babc-319bc5b5af95 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:15.422996 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa582ae9-d5b7-4da7-97c8-e0abc19f46b6 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:15.432807 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40997f0b-52a3-44a6-babc-319bc5b5af95 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:15.441219 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:15.488253 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a356bb-3094-4442-a796-b4b0111e98f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:15.489665 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a356bb-3094-4442-a796-b4b0111e98f5 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:15.489944 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a356bb-3094-4442-a796-b4b0111e98f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:15.490260 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a356bb-3094-4442-a796-b4b0111e98f5 None admin] PATH_INFO: `/v3/users/9b2edac556674677ba6894dc26bd2aab` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:15.491424 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a83325-5067-4af0-a469-670ca6f105e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:15.492097 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a83325-5067-4af0-a469-670ca6f105e1 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:15.492362 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a83325-5067-4af0-a469-670ca6f105e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:15.492635 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a83325-5067-4af0-a469-670ca6f105e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:15.493482 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a83325-5067-4af0-a469-670ca6f105e1 None admin] RBAC: Adding query filter params (role.id=c5431c9deadf47e496264ba5b78eeaf6, scope.project.id=b1e275c904b64687b29e7ca3d28fdda7, user.id=9b2edac556674677ba6894dc26bd2aab) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:15.493792 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a83325-5067-4af0-a469-670ca6f105e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:15.495279 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a83325-5067-4af0-a469-670ca6f105e1 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:15.497823 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a356bb-3094-4442-a796-b4b0111e98f5 None admin] RBAC: Authorizing `identity:get_user(user_id=9b2edac556674677ba6894dc26bd2aab)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:15.499288 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a356bb-3094-4442-a796-b4b0111e98f5 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:15.501312 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 83/170] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:49:15 2022] GET /identity/v3/users/9b2edac556674677ba6894dc26bd2aab => generated 275 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:15.501830 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a356bb-3094-4442-a796-b4b0111e98f5 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:15.513395 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 88/171] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:49:15 2022] GET /identity/v3/role_assignments?user.id=9b2edac556674677ba6894dc26bd2aab&scope.project.id=b1e275c904b64687b29e7ca3d28fdda7&role.id=c5431c9deadf47e496264ba5b78eeaf6 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:15.513984 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a83325-5067-4af0-a469-670ca6f105e1 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:15.765256 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a356bb-3094-4442-a796-b4b0111e98f5 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:15.775661 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:15.808642 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b5f795-ac2f-41a6-9b5b-6c22500b06c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:15.809564 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5f795-ac2f-41a6-9b5b-6c22500b06c2 None admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:15.809848 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5f795-ac2f-41a6-9b5b-6c22500b06c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:15.810123 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5f795-ac2f-41a6-9b5b-6c22500b06c2 None admin] PATH_INFO: `/v3/projects/b1e275c904b64687b29e7ca3d28fdda7/users/8f1f1a21224a4f3093848cab630ffa09/roles/30b0c2ba85ff481eb256122140e4163a` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:15.812245 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b5f795-ac2f-41a6-9b5b-6c22500b06c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=b1e275c904b64687b29e7ca3d28fdda7, user_id=8f1f1a21224a4f3093848cab630ffa09, role_id=30b0c2ba85ff481eb256122140e4163a)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:15.814439 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b5f795-ac2f-41a6-9b5b-6c22500b06c2 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:15.837403 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a83325-5067-4af0-a469-670ca6f105e1 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:15.840636 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 84/172] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jun 7 21:49:15 2022] PUT /identity/v3/projects/b1e275c904b64687b29e7ca3d28fdda7/users/8f1f1a21224a4f3093848cab630ffa09/roles/30b0c2ba85ff481eb256122140e4163a => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 21:49:15.848513 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b5f795-ac2f-41a6-9b5b-6c22500b06c2 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:15.963044 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fbae5a-c8e3-4667-9863-cfde89caa1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:15.963769 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fbae5a-c8e3-4667-9863-cfde89caa1d6 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:15.964058 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fbae5a-c8e3-4667-9863-cfde89caa1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:15.964340 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fbae5a-c8e3-4667-9863-cfde89caa1d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:15.965125 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fbae5a-c8e3-4667-9863-cfde89caa1d6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:15.965440 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fbae5a-c8e3-4667-9863-cfde89caa1d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:15.966900 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fbae5a-c8e3-4667-9863-cfde89caa1d6 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:15.996589 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 89/173] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jun 7 21:49:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 483 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:15.999072 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b5f795-ac2f-41a6-9b5b-6c22500b06c2 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:16.004495 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fbae5a-c8e3-4667-9863-cfde89caa1d6 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:16.045307 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f32ea-8d67-4840-bc83-9ba325b0ad24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:16.045987 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f32ea-8d67-4840-bc83-9ba325b0ad24 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:16.046279 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f32ea-8d67-4840-bc83-9ba325b0ad24 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:16.046567 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f32ea-8d67-4840-bc83-9ba325b0ad24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:16.099793 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:16.111541 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:16.136416 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6f32ea-8d67-4840-bc83-9ba325b0ad24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:16.137640 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6f32ea-8d67-4840-bc83-9ba325b0ad24 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:16.279796 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 85/174] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:49:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 21:49:16.285146 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f32ea-8d67-4840-bc83-9ba325b0ad24 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:16.342804 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fbae5a-c8e3-4667-9863-cfde89caa1d6 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:16.383077 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:16.383260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8623e6d-a101-48d8-9409-f152f2daa061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:16.384091 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8623e6d-a101-48d8-9409-f152f2daa061 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:16.384376 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8623e6d-a101-48d8-9409-f152f2daa061 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:16.384649 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8623e6d-a101-48d8-9409-f152f2daa061 None admin] PATH_INFO: `/v3/users/9b2edac556674677ba6894dc26bd2aab` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:16.386008 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8623e6d-a101-48d8-9409-f152f2daa061 None admin] RBAC: Authorizing `identity:get_user(user_id=9b2edac556674677ba6894dc26bd2aab)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:16.387515 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8623e6d-a101-48d8-9409-f152f2daa061 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:16.389199 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 90/175] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:49:16 2022] GET /identity/v3/users/9b2edac556674677ba6894dc26bd2aab => generated 275 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:16.390395 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c5ef9-fe2f-4e5b-b3de-9adc4d66ff9a None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:16.390684 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c5ef9-fe2f-4e5b-b3de-9adc4d66ff9a None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:16.390968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c5ef9-fe2f-4e5b-b3de-9adc4d66ff9a None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:16.397601 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 91/176] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:16.398372 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644c5ef9-fe2f-4e5b-b3de-9adc4d66ff9a None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:16.414781 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f32ea-8d67-4840-bc83-9ba325b0ad24 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:16.458365 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfb947a-29c4-4a83-9853-ad276b0d6b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:16.459358 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb947a-29c4-4a83-9853-ad276b0d6b23 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:16.459640 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb947a-29c4-4a83-9853-ad276b0d6b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:16.459989 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb947a-29c4-4a83-9853-ad276b0d6b23 None admin] PATH_INFO: `/v3/projects/b1e275c904b64687b29e7ca3d28fdda7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:16.461450 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfb947a-29c4-4a83-9853-ad276b0d6b23 None admin] RBAC: Authorizing `identity:get_project(project_id=b1e275c904b64687b29e7ca3d28fdda7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:16.462990 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfb947a-29c4-4a83-9853-ad276b0d6b23 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:16.464737 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 86/177] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:49:16 2022] GET /identity/v3/projects/b1e275c904b64687b29e7ca3d28fdda7 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:49:16.466055 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc50c5f-9fba-4208-8142-634bc9be384f None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:16.466349 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc50c5f-9fba-4208-8142-634bc9be384f None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:16.466746 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc50c5f-9fba-4208-8142-634bc9be384f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:16.485564 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-1bc50c5f-9fba-4208-8142-634bc9be384f None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:16.486802 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-1bc50c5f-9fba-4208-8142-634bc9be384f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:16.559445 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 87/178] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:49:16.560430 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc50c5f-9fba-4208-8142-634bc9be384f None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:16.583953 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:16.601285 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644c5ef9-fe2f-4e5b-b3de-9adc4d66ff9a None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:16.634840 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:16.672641 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfbcff4-f747-4f55-91a3-dcee62279126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:16.672955 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc50c5f-9fba-4208-8142-634bc9be384f None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:16.673311 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfbcff4-f747-4f55-91a3-dcee62279126 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:16.673607 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfbcff4-f747-4f55-91a3-dcee62279126 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:16.673866 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfbcff4-f747-4f55-91a3-dcee62279126 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:16.719009 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc9d29-61a4-4eff-ae53-ae16ba1b2d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:16.720063 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc9d29-61a4-4eff-ae53-ae16ba1b2d38 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:16.720551 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc9d29-61a4-4eff-ae53-ae16ba1b2d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:16.721001 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc9d29-61a4-4eff-ae53-ae16ba1b2d38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:16.722269 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdc9d29-61a4-4eff-ae53-ae16ba1b2d38 None admin] RBAC: Adding query filter params (role.id=50e04e9d09e045b0bcf8e1782a6e2f23, scope.project.id=b1e275c904b64687b29e7ca3d28fdda7, user.id=9b2edac556674677ba6894dc26bd2aab) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:16.722980 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdc9d29-61a4-4eff-ae53-ae16ba1b2d38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:16.725030 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdc9d29-61a4-4eff-ae53-ae16ba1b2d38 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:16.732437 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 88/179] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:49:16 2022] GET /identity/v3/role_assignments?user.id=9b2edac556674677ba6894dc26bd2aab&scope.project.id=b1e275c904b64687b29e7ca3d28fdda7&role.id=50e04e9d09e045b0bcf8e1782a6e2f23 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:16.733751 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc9d29-61a4-4eff-ae53-ae16ba1b2d38 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:16.824524 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfbcff4-f747-4f55-91a3-dcee62279126 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:16.826129 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfbcff4-f747-4f55-91a3-dcee62279126 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:16.854054 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc9d29-61a4-4eff-ae53-ae16ba1b2d38 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:16.884464 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:16.894280 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fe799f-4368-4584-aa67-b9c6c99531cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:16.896405 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe799f-4368-4584-aa67-b9c6c99531cc None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:16.896897 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe799f-4368-4584-aa67-b9c6c99531cc None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:16.897448 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe799f-4368-4584-aa67-b9c6c99531cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:16.909501 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fe799f-4368-4584-aa67-b9c6c99531cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:16.918378 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-d7fe799f-4368-4584-aa67-b9c6c99531cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:49:16.918378 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:16.918378 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:16.918378 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:16.918378 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:16.918378 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:16.918378 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:16.918378 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:16.918378 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:16.918378 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:16.918378 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:16.918378 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:16.918378 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:16.918378 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:16.918378 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:16.918378 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:16.919365 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:16.919365 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:16.919365 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:16.919365 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:16.919365 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:16.919365 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:16.919365 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:16.919365 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:16.919365 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:16.919365 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:16.919365 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:16.919365 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:16.919365 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:16.919365 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:16.919365 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:16.919365 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:16.920195 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:16.920195 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:16.920195 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:16.920195 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:16.920195 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:16.920195 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:16.920195 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:16.920195 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:16.920195 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:16.920195 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:16.920195 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:49:16.920195 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:49:16.922773 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 89/180] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jun 7 21:49:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:49:16.932522 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fe799f-4368-4584-aa67-b9c6c99531cc None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:17.023513 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:17.032340 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 92/181] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:49:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 634 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 21:49:17.037933 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfbcff4-f747-4f55-91a3-dcee62279126 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:17.050182 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fe799f-4368-4584-aa67-b9c6c99531cc None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:17.090752 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a39d72-7af0-4488-8d71-509995f53fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:17.091956 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a39d72-7af0-4488-8d71-509995f53fc6 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:17.092549 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a39d72-7af0-4488-8d71-509995f53fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:17.093031 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a39d72-7af0-4488-8d71-509995f53fc6 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:17.093998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a39d72-7af0-4488-8d71-509995f53fc6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:17.094529 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a39d72-7af0-4488-8d71-509995f53fc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:17.096242 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a39d72-7af0-4488-8d71-509995f53fc6 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:17.106900 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 90/182] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:49:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:49:17.112333 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a39d72-7af0-4488-8d71-509995f53fc6 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:17.219689 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:17.229637 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfbcff4-f747-4f55-91a3-dcee62279126 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:17.270602 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c29699b-22ec-4e82-b2fa-8e531405634e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:17.271796 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29699b-22ec-4e82-b2fa-8e531405634e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:17.272360 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29699b-22ec-4e82-b2fa-8e531405634e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:17.272468 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29699b-22ec-4e82-b2fa-8e531405634e None admin] PATH_INFO: `/v3/projects/0fa85257194347128519249f6161fd94` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:17.273687 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c29699b-22ec-4e82-b2fa-8e531405634e None admin] RBAC: Authorizing `identity:get_project(project_id=0fa85257194347128519249f6161fd94)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:17.275207 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c29699b-22ec-4e82-b2fa-8e531405634e None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:17.277370 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 93/183] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:49:17 2022] GET /identity/v3/projects/0fa85257194347128519249f6161fd94 => generated 312 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:49:17.278639 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18798a-fb8d-412f-b972-5da1d72df5d3 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:17.278930 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18798a-fb8d-412f-b972-5da1d72df5d3 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:17.279205 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18798a-fb8d-412f-b972-5da1d72df5d3 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:17.279928 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a39d72-7af0-4488-8d71-509995f53fc6 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:17.280471 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 94/184] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:17.284205 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c18798a-fb8d-412f-b972-5da1d72df5d3 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:17.323855 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe2c6df-c946-4cdc-91fc-d5a227d44303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:17.324755 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe2c6df-c946-4cdc-91fc-d5a227d44303 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:17.325045 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe2c6df-c946-4cdc-91fc-d5a227d44303 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:17.325327 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe2c6df-c946-4cdc-91fc-d5a227d44303 None admin] PATH_INFO: `/v3/users/b80029c598ad44d4a8eaf3e161a8bb5b` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:17.342907 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe2c6df-c946-4cdc-91fc-d5a227d44303 None admin] RBAC: Authorizing `identity:get_user(user_id=b80029c598ad44d4a8eaf3e161a8bb5b)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:17.344415 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe2c6df-c946-4cdc-91fc-d5a227d44303 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:17.346152 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 91/185] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:49:17 2022] GET /identity/v3/users/b80029c598ad44d4a8eaf3e161a8bb5b => generated 283 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:17.347820 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae3161-282b-4991-829c-1cfcf6483245 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:17.348401 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae3161-282b-4991-829c-1cfcf6483245 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:17.348758 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae3161-282b-4991-829c-1cfcf6483245 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:17.350031 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 92/186] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:17.352322 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd51623-129e-42ec-8bef-03f4e0ea2e4d None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:17.352612 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd51623-129e-42ec-8bef-03f4e0ea2e4d None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:17.352894 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd51623-129e-42ec-8bef-03f4e0ea2e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:17.365703 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:17.371921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-acd51623-129e-42ec-8bef-03f4e0ea2e4d None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:17.373122 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-acd51623-129e-42ec-8bef-03f4e0ea2e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:17.408345 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c18798a-fb8d-412f-b972-5da1d72df5d3 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:17.451351 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 93/187] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:49:17.451351 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd51623-129e-42ec-8bef-03f4e0ea2e4d None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:17.454876 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419e769a-0ecb-4ed4-8efb-f0884a86f5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:17.455672 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419e769a-0ecb-4ed4-8efb-f0884a86f5d4 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:17.456058 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419e769a-0ecb-4ed4-8efb-f0884a86f5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:17.456456 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419e769a-0ecb-4ed4-8efb-f0884a86f5d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:17.457947 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419e769a-0ecb-4ed4-8efb-f0884a86f5d4 None admin] RBAC: Adding query filter params (role.id=c5431c9deadf47e496264ba5b78eeaf6, scope.project.id=0fa85257194347128519249f6161fd94, user.id=9b2edac556674677ba6894dc26bd2aab) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:17.458336 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419e769a-0ecb-4ed4-8efb-f0884a86f5d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:17.460068 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419e769a-0ecb-4ed4-8efb-f0884a86f5d4 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:17.466862 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 95/188] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:49:17 2022] GET /identity/v3/role_assignments?user.id=9b2edac556674677ba6894dc26bd2aab&scope.project.id=0fa85257194347128519249f6161fd94&role.id=c5431c9deadf47e496264ba5b78eeaf6 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:17.473292 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880464f2-9138-4bb1-a26c-0b88bf6f5895 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:17.473584 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880464f2-9138-4bb1-a26c-0b88bf6f5895 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:17.473863 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880464f2-9138-4bb1-a26c-0b88bf6f5895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:17.505630 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-880464f2-9138-4bb1-a26c-0b88bf6f5895 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:17.507767 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:17.507971 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-880464f2-9138-4bb1-a26c-0b88bf6f5895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:17.566678 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd51623-129e-42ec-8bef-03f4e0ea2e4d None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:17.605864 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3aa5ec-c271-44ba-92f4-dcb4fa7e24d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:17.606943 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3aa5ec-c271-44ba-92f4-dcb4fa7e24d0 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:17.607481 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3aa5ec-c271-44ba-92f4-dcb4fa7e24d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:17.607955 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3aa5ec-c271-44ba-92f4-dcb4fa7e24d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:17.608647 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 96/189] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:49:17.609696 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880464f2-9138-4bb1-a26c-0b88bf6f5895 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:17.685974 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:17.709019 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3aa5ec-c271-44ba-92f4-dcb4fa7e24d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:17.710501 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3aa5ec-c271-44ba-92f4-dcb4fa7e24d0 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:17.738365 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880464f2-9138-4bb1-a26c-0b88bf6f5895 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:17.765082 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:17.793753 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a378823-9e70-4311-a174-d1981866be56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:17.794864 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a378823-9e70-4311-a174-d1981866be56 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:17.795148 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a378823-9e70-4311-a174-d1981866be56 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:17.795473 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a378823-9e70-4311-a174-d1981866be56 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:17.810381 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a378823-9e70-4311-a174-d1981866be56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:17.822010 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-0a378823-9e70-4311-a174-d1981866be56 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 21:49:17.822010 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:17.822010 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:17.822010 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:17.822010 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:17.822010 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:17.822010 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:17.822010 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:17.822010 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:17.822010 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:17.822010 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:17.822010 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:17.822010 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:17.822010 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:17.822010 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:17.822010 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:17.822877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:17.822877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:17.822877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:17.822877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:17.822877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:17.822877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:17.822877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:17.822877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:17.822877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:17.822877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:17.822877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:17.822877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:17.822877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:17.822877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:17.822877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:17.822877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:17.823895 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:17.823895 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:17.823895 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:17.823895 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:17.823895 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:17.823895 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:17.823895 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:17.823895 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:17.823895 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:17.823895 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:17.823895 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 21:49:17.823895 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:49:17.823895 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 97/190] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Jun 7 21:49:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:49:17.824876 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a378823-9e70-4311-a174-d1981866be56 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:17.840515 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 94/191] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:49:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 21:49:17.841214 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3aa5ec-c271-44ba-92f4-dcb4fa7e24d0 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:17.928580 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a378823-9e70-4311-a174-d1981866be56 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:17.945642 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:17.953675 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3aa5ec-c271-44ba-92f4-dcb4fa7e24d0 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:17.964662 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b35c01-0f33-48f8-b643-fc8a8e2aaa5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:17.965661 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b35c01-0f33-48f8-b643-fc8a8e2aaa5d None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:17.966127 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b35c01-0f33-48f8-b643-fc8a8e2aaa5d None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:17.966465 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b35c01-0f33-48f8-b643-fc8a8e2aaa5d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:17.967723 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b35c01-0f33-48f8-b643-fc8a8e2aaa5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:17.969211 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b35c01-0f33-48f8-b643-fc8a8e2aaa5d None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:17.975863 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 98/192] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:49:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:17.975863 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b35c01-0f33-48f8-b643-fc8a8e2aaa5d None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:17.995491 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015db970-25cf-402b-82a6-7df5c39e46f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:17.996172 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015db970-25cf-402b-82a6-7df5c39e46f2 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:17.996456 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015db970-25cf-402b-82a6-7df5c39e46f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:17.996773 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015db970-25cf-402b-82a6-7df5c39e46f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:17.997916 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015db970-25cf-402b-82a6-7df5c39e46f2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:17.998383 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015db970-25cf-402b-82a6-7df5c39e46f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:17.999814 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015db970-25cf-402b-82a6-7df5c39e46f2 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:18.001990 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:18.008863 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 95/193] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:49:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:18.009734 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015db970-25cf-402b-82a6-7df5c39e46f2 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:18.086960 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b35c01-0f33-48f8-b643-fc8a8e2aaa5d None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:18.117548 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015db970-25cf-402b-82a6-7df5c39e46f2 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:18.125361 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91b9703-2c69-4001-a2ea-750477e9e787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:18.126230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b9703-2c69-4001-a2ea-750477e9e787 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:18.126880 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b9703-2c69-4001-a2ea-750477e9e787 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:18.127162 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b9703-2c69-4001-a2ea-750477e9e787 None admin] PATH_INFO: `/v3/projects/e8ca129f01d149b5aab391353f90139b` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:18.128683 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91b9703-2c69-4001-a2ea-750477e9e787 None admin] RBAC: Authorizing `identity:get_project(project_id=e8ca129f01d149b5aab391353f90139b)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:18.130278 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91b9703-2c69-4001-a2ea-750477e9e787 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:18.132202 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 99/194] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:49:17 2022] GET /identity/v3/projects/e8ca129f01d149b5aab391353f90139b => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:18.132961 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91b9703-2c69-4001-a2ea-750477e9e787 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:18.159456 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5478e-0a28-4314-b084-11d2404e4c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:18.161100 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5478e-0a28-4314-b084-11d2404e4c7d None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:18.161802 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5478e-0a28-4314-b084-11d2404e4c7d None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:18.162275 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5478e-0a28-4314-b084-11d2404e4c7d None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:18.163285 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f5478e-0a28-4314-b084-11d2404e4c7d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:18.164987 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f5478e-0a28-4314-b084-11d2404e4c7d None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:18.189003 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 96/195] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Jun 7 21:49:18 2022] POST /identity/v3/users => generated 297 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 21:49:18.190484 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5478e-0a28-4314-b084-11d2404e4c7d None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:18.196581 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:18.256194 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91b9703-2c69-4001-a2ea-750477e9e787 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:18.276757 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:18.304509 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abb504d-ee64-409c-a750-bc9129677e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:18.305423 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb504d-ee64-409c-a750-bc9129677e03 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:18.305709 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb504d-ee64-409c-a750-bc9129677e03 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:18.305989 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb504d-ee64-409c-a750-bc9129677e03 None admin] PATH_INFO: `/v3/users/b80029c598ad44d4a8eaf3e161a8bb5b` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:18.307338 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abb504d-ee64-409c-a750-bc9129677e03 None admin] RBAC: Authorizing `identity:get_user(user_id=b80029c598ad44d4a8eaf3e161a8bb5b)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:18.308882 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abb504d-ee64-409c-a750-bc9129677e03 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:18.311150 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 100/196] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:49:18 2022] GET /identity/v3/users/b80029c598ad44d4a8eaf3e161a8bb5b => generated 283 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:18.316808 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abb504d-ee64-409c-a750-bc9129677e03 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:18.338638 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5478e-0a28-4314-b084-11d2404e4c7d None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:18.378445 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d1cb8d-53b0-4a41-b718-ca2da647375a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:18.380053 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1cb8d-53b0-4a41-b718-ca2da647375a None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:18.380755 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1cb8d-53b0-4a41-b718-ca2da647375a None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:18.381269 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1cb8d-53b0-4a41-b718-ca2da647375a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:18.382401 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d1cb8d-53b0-4a41-b718-ca2da647375a None admin] RBAC: Adding query filter params (role.id=6b483758dbd44eb4a78df2ce4603d601, scope.project.id=e8ca129f01d149b5aab391353f90139b, user.id=b80029c598ad44d4a8eaf3e161a8bb5b) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:18.382918 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d1cb8d-53b0-4a41-b718-ca2da647375a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:18.384730 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d1cb8d-53b0-4a41-b718-ca2da647375a None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:18.392014 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 97/197] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:49:18 2022] GET /identity/v3/role_assignments?user.id=b80029c598ad44d4a8eaf3e161a8bb5b&scope.project.id=e8ca129f01d149b5aab391353f90139b&role.id=6b483758dbd44eb4a78df2ce4603d601 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:18.433905 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abb504d-ee64-409c-a750-bc9129677e03 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:18.460592 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:18.477148 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626014c0-7873-4e6d-ae14-4d48210bdb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:18.477815 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626014c0-7873-4e6d-ae14-4d48210bdb75 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:18.478151 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626014c0-7873-4e6d-ae14-4d48210bdb75 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:18.478434 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626014c0-7873-4e6d-ae14-4d48210bdb75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:18.560041 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626014c0-7873-4e6d-ae14-4d48210bdb75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:18.561538 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626014c0-7873-4e6d-ae14-4d48210bdb75 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:18.673652 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 101/198] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:49:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 21:49:18.680460 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d1cb8d-53b0-4a41-b718-ca2da647375a None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:18.796874 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d1cb8d-53b0-4a41-b718-ca2da647375a None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:18.814607 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:18.838082 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ecf11f-256b-43e9-ad7d-005ed653a01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:18.838971 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ecf11f-256b-43e9-ad7d-005ed653a01c None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:18.839252 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ecf11f-256b-43e9-ad7d-005ed653a01c None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:18.839580 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ecf11f-256b-43e9-ad7d-005ed653a01c None admin] PATH_INFO: `/v3/projects/e8ca129f01d149b5aab391353f90139b` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:18.842680 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ecf11f-256b-43e9-ad7d-005ed653a01c None admin] RBAC: Authorizing `identity:get_project(project_id=e8ca129f01d149b5aab391353f90139b)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:18.844247 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ecf11f-256b-43e9-ad7d-005ed653a01c None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:18.846328 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 98/199] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:49:18 2022] GET /identity/v3/projects/e8ca129f01d149b5aab391353f90139b => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:18.853090 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626014c0-7873-4e6d-ae14-4d48210bdb75 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:18.875299 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd24a3f-93dd-483e-835c-710f217a4007 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:18.875634 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd24a3f-93dd-483e-835c-710f217a4007 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:18.875881 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd24a3f-93dd-483e-835c-710f217a4007 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:18.877260 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 99/200] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 21:49:18.883215 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c804887-a62b-4cc3-9665-e96297545e95 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:18.883544 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c804887-a62b-4cc3-9665-e96297545e95 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:18.883783 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:18.883889 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c804887-a62b-4cc3-9665-e96297545e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:18.902155 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-3c804887-a62b-4cc3-9665-e96297545e95 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:18.903370 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-3c804887-a62b-4cc3-9665-e96297545e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:18.984034 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 100/201] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 21:49:18.991275 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626014c0-7873-4e6d-ae14-4d48210bdb75 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:19.009808 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c804887-a62b-4cc3-9665-e96297545e95 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:19.039969 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977fe5b6-c37b-4a00-b648-1a6be129bd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:19.040637 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977fe5b6-c37b-4a00-b648-1a6be129bd0c None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:19.040921 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977fe5b6-c37b-4a00-b648-1a6be129bd0c None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:19.041207 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977fe5b6-c37b-4a00-b648-1a6be129bd0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:19.042057 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977fe5b6-c37b-4a00-b648-1a6be129bd0c None admin] RBAC: Adding query filter params (role.id=50e04e9d09e045b0bcf8e1782a6e2f23, scope.project.id=e8ca129f01d149b5aab391353f90139b, user.id=b80029c598ad44d4a8eaf3e161a8bb5b) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:19.042368 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977fe5b6-c37b-4a00-b648-1a6be129bd0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:19.045158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977fe5b6-c37b-4a00-b648-1a6be129bd0c None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:19.055899 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 102/202] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:49:18 2022] GET /identity/v3/role_assignments?user.id=b80029c598ad44d4a8eaf3e161a8bb5b&scope.project.id=e8ca129f01d149b5aab391353f90139b&role.id=50e04e9d09e045b0bcf8e1782a6e2f23 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:19.151348 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c804887-a62b-4cc3-9665-e96297545e95 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:19.182443 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:19.210783 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c9710-31fe-4c14-9b88-b572b35086b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:19.211639 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c9710-31fe-4c14-9b88-b572b35086b5 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:19.211917 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c9710-31fe-4c14-9b88-b572b35086b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:19.212192 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c9710-31fe-4c14-9b88-b572b35086b5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:19.234720 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338c9710-31fe-4c14-9b88-b572b35086b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:19.243368 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-338c9710-31fe-4c14-9b88-b572b35086b5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 21:49:19.243368 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:19.243368 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:19.243368 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:19.243368 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:19.243368 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:19.243368 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:19.243368 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:19.243368 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:19.243368 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:19.243368 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:19.243368 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:19.243368 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:19.243368 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:19.243368 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:19.243368 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:19.244347 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:19.244347 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:19.244347 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:19.244347 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:19.244347 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:19.244347 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:19.244347 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:19.244347 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:19.244347 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:19.244347 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:19.244347 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:19.244347 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:19.244347 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:19.244347 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:19.244347 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:19.244347 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:19.245147 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:19.245147 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:19.245147 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:19.245147 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:19.245147 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:19.245147 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:19.245147 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:19.245147 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:19.245147 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:19.245147 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:19.245147 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 21:49:19.245147 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:49:19.245147 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 101/203] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:49:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:49:19.250684 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977fe5b6-c37b-4a00-b648-1a6be129bd0c None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:19.384978 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:19.402835 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977fe5b6-c37b-4a00-b648-1a6be129bd0c None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:19.438453 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6310355a-fb37-4f6a-b1a9-b85bbea01b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:19.439125 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310355a-fb37-4f6a-b1a9-b85bbea01b5e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:19.439438 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310355a-fb37-4f6a-b1a9-b85bbea01b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:19.439728 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310355a-fb37-4f6a-b1a9-b85bbea01b5e None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:19.440487 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6310355a-fb37-4f6a-b1a9-b85bbea01b5e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:19.440798 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6310355a-fb37-4f6a-b1a9-b85bbea01b5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:19.442718 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6310355a-fb37-4f6a-b1a9-b85bbea01b5e None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:19.455453 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 103/204] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jun 7 21:49:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:19.460406 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c9710-31fe-4c14-9b88-b572b35086b5 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:19.599837 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c9710-31fe-4c14-9b88-b572b35086b5 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:19.649992 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad71dcf-bd4a-4eee-8155-a4c426c2d2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:19.651190 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad71dcf-bd4a-4eee-8155-a4c426c2d2c2 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:19.651676 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad71dcf-bd4a-4eee-8155-a4c426c2d2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:19.652187 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad71dcf-bd4a-4eee-8155-a4c426c2d2c2 None admin] PATH_INFO: `/v3/users/8f1f1a21224a4f3093848cab630ffa09` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:19.653851 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad71dcf-bd4a-4eee-8155-a4c426c2d2c2 None admin] RBAC: Authorizing `identity:get_user(user_id=8f1f1a21224a4f3093848cab630ffa09)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:19.655567 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad71dcf-bd4a-4eee-8155-a4c426c2d2c2 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:19.658653 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 102/205] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:49:19 2022] GET /identity/v3/users/8f1f1a21224a4f3093848cab630ffa09 => generated 289 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:19.664015 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6310355a-fb37-4f6a-b1a9-b85bbea01b5e None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:19.785018 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6310355a-fb37-4f6a-b1a9-b85bbea01b5e None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:19.830828 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0530cb49-00bf-48f2-92e0-5047071b651e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:19.831518 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0530cb49-00bf-48f2-92e0-5047071b651e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:19.831801 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0530cb49-00bf-48f2-92e0-5047071b651e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:19.832583 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0530cb49-00bf-48f2-92e0-5047071b651e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:19.858993 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:19.929043 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0530cb49-00bf-48f2-92e0-5047071b651e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:19.930693 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0530cb49-00bf-48f2-92e0-5047071b651e None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:20.075156 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 104/206] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:49:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 21:49:20.081140 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad71dcf-bd4a-4eee-8155-a4c426c2d2c2 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:20.149919 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:20.216162 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad71dcf-bd4a-4eee-8155-a4c426c2d2c2 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:20.251650 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b3286b-4938-41be-bbed-ae6f4c45069c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:20.252516 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b3286b-4938-41be-bbed-ae6f4c45069c None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:20.252799 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b3286b-4938-41be-bbed-ae6f4c45069c None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:20.253076 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b3286b-4938-41be-bbed-ae6f4c45069c None admin] PATH_INFO: `/v3/projects/b1e275c904b64687b29e7ca3d28fdda7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:20.254795 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b3286b-4938-41be-bbed-ae6f4c45069c None admin] RBAC: Authorizing `identity:get_project(project_id=b1e275c904b64687b29e7ca3d28fdda7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:20.256270 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b3286b-4938-41be-bbed-ae6f4c45069c None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:20.258281 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 103/207] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:49:20 2022] GET /identity/v3/projects/b1e275c904b64687b29e7ca3d28fdda7 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:20.264105 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0530cb49-00bf-48f2-92e0-5047071b651e None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:20.339723 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:20.380604 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0530cb49-00bf-48f2-92e0-5047071b651e None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:20.415926 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc58dba-f4e2-4c55-b6e5-e3a82c636638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:20.416580 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc58dba-f4e2-4c55-b6e5-e3a82c636638 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:20.416864 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc58dba-f4e2-4c55-b6e5-e3a82c636638 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:20.417138 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc58dba-f4e2-4c55-b6e5-e3a82c636638 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:20.417974 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc58dba-f4e2-4c55-b6e5-e3a82c636638 None admin] RBAC: Adding query filter params (role.id=30b0c2ba85ff481eb256122140e4163a, scope.project.id=b1e275c904b64687b29e7ca3d28fdda7, user.id=8f1f1a21224a4f3093848cab630ffa09) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:20.418284 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc58dba-f4e2-4c55-b6e5-e3a82c636638 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:20.419754 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc58dba-f4e2-4c55-b6e5-e3a82c636638 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:20.425642 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 105/208] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:49:20 2022] GET /identity/v3/role_assignments?user.id=8f1f1a21224a4f3093848cab630ffa09&scope.project.id=b1e275c904b64687b29e7ca3d28fdda7&role.id=30b0c2ba85ff481eb256122140e4163a => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:21.175384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94cb7c4-369d-4348-b839-1d6821b2a3bc None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:21.176532 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94cb7c4-369d-4348-b839-1d6821b2a3bc None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:21.176688 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94cb7c4-369d-4348-b839-1d6821b2a3bc None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:21.178000 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 104/209] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:21.183414 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f1019-d55a-4895-b5c3-f05eef6013b0 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:21.183816 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f1019-d55a-4895-b5c3-f05eef6013b0 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:21.183984 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f1019-d55a-4895-b5c3-f05eef6013b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:21.201085 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-695f1019-d55a-4895-b5c3-f05eef6013b0 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:21.202267 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-695f1019-d55a-4895-b5c3-f05eef6013b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:21.275658 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 106/210] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:49:21.294371 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e990b6-c757-40b3-8b9e-9af275fb65c1 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:21.294908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cabb570-8778-4582-a9e4-dec8d0882325 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:21.295203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cabb570-8778-4582-a9e4-dec8d0882325 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:21.295460 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e990b6-c757-40b3-8b9e-9af275fb65c1 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:21.295560 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cabb570-8778-4582-a9e4-dec8d0882325 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:21.296026 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e990b6-c757-40b3-8b9e-9af275fb65c1 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:21.296963 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 107/211] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 21:49:21.298285 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cabb570-8778-4582-a9e4-dec8d0882325 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:21.298520 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 105/212] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:21.302950 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd36b7-8312-4df6-9cc1-3e5017892090 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:21.303266 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd36b7-8312-4df6-9cc1-3e5017892090 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:21.303592 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd36b7-8312-4df6-9cc1-3e5017892090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:21.322282 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-75bd36b7-8312-4df6-9cc1-3e5017892090 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:21.323519 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-75bd36b7-8312-4df6-9cc1-3e5017892090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:21.348540 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:21.399481 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 106/213] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 21:49:21.400855 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece795c-194d-451e-b8ec-514f837771ef None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:21.401148 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece795c-194d-451e-b8ec-514f837771ef None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:21.401427 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece795c-194d-451e-b8ec-514f837771ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:21.412826 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cabb570-8778-4582-a9e4-dec8d0882325 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:21.420146 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-0ece795c-194d-451e-b8ec-514f837771ef None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:21.421645 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-0ece795c-194d-451e-b8ec-514f837771ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:21.451318 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df3c67f-d008-49e6-9c65-a55547083d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:21.452073 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df3c67f-d008-49e6-9c65-a55547083d8a None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:21.452357 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df3c67f-d008-49e6-9c65-a55547083d8a None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:21.452633 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df3c67f-d008-49e6-9c65-a55547083d8a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:21.460886 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:21.464023 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df3c67f-d008-49e6-9c65-a55547083d8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:21.472021 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-0df3c67f-d008-49e6-9c65-a55547083d8a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 21:49:21.472021 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:21.472021 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:21.472021 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:21.472021 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:21.472021 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:21.472021 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:21.472021 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:21.472021 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:21.472021 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:21.472021 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:21.472021 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:21.472021 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:21.472021 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:21.472021 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:21.472021 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:21.473444 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:21.473444 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:21.473444 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:21.473444 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:21.473444 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:21.473444 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:21.473444 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:21.473444 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:21.473444 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:21.473444 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:21.473444 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:21.473444 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:21.473444 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:21.473444 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:21.473444 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:21.473444 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:21.474249 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:21.474249 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:21.474249 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:21.474249 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:21.474249 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:21.474249 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:21.474249 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:21.474249 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:21.474249 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:21.474249 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:21.474249 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 21:49:21.474249 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:49:21.474249 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 108/214] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:49:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:49:21.474921 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df3c67f-d008-49e6-9c65-a55547083d8a None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:21.503980 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 107/215] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:49:21.504746 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece795c-194d-451e-b8ec-514f837771ef None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:21.588925 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df3c67f-d008-49e6-9c65-a55547083d8a None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:21.590608 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:21.623621 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece795c-194d-451e-b8ec-514f837771ef None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:21.633562 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba8c4d-7249-40f9-b9f5-837131cc3b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:21.634391 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba8c4d-7249-40f9-b9f5-837131cc3b01 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:21.634678 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba8c4d-7249-40f9-b9f5-837131cc3b01 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:21.634954 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba8c4d-7249-40f9-b9f5-837131cc3b01 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:21.646971 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba8c4d-7249-40f9-b9f5-837131cc3b01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:21.654700 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-8aba8c4d-7249-40f9-b9f5-837131cc3b01 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:49:21.654700 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:21.654700 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:21.654700 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:21.654700 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:21.654700 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:21.654700 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:21.654700 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:21.654700 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:21.654700 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:21.654700 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:21.654700 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:21.654700 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:21.654700 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:21.654700 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:21.654700 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:21.655663 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:21.655663 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:21.655663 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:21.655663 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:21.655663 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:21.655663 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:21.655663 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:21.655663 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:21.655663 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:21.655663 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:21.655663 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:21.655663 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:21.655663 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:21.655663 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:21.655663 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:21.655663 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:21.656549 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:21.656549 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:21.656549 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:21.656549 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:21.656549 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:21.656549 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:21.656549 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:21.656549 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:21.656549 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:21.656549 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:21.656549 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:49:21.656549 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:49:21.656549 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 109/216] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jun 7 21:49:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:49:21.657209 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba8c4d-7249-40f9-b9f5-837131cc3b01 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:21.668473 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74178c-c00d-4770-93da-301b1d2f4462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:21.669176 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74178c-c00d-4770-93da-301b1d2f4462 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:21.669464 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74178c-c00d-4770-93da-301b1d2f4462 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:21.669752 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74178c-c00d-4770-93da-301b1d2f4462 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:21.670534 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb74178c-c00d-4770-93da-301b1d2f4462 None admin] RBAC: Adding query filter params (name=member) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:21.670816 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb74178c-c00d-4770-93da-301b1d2f4462 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:21.672281 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb74178c-c00d-4770-93da-301b1d2f4462 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:21.681654 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 108/217] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jun 7 21:49:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:21.686757 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74178c-c00d-4770-93da-301b1d2f4462 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:21.723503 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:21.775870 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba8c4d-7249-40f9-b9f5-837131cc3b01 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:21.798974 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74178c-c00d-4770-93da-301b1d2f4462 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:21.811049 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f4f44-f5f1-4f07-905f-ac08653ca59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:21.811891 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f4f44-f5f1-4f07-905f-ac08653ca59f None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:21.812177 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f4f44-f5f1-4f07-905f-ac08653ca59f None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:21.812464 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f4f44-f5f1-4f07-905f-ac08653ca59f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:21.813634 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7f4f44-f5f1-4f07-905f-ac08653ca59f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:21.815079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7f4f44-f5f1-4f07-905f-ac08653ca59f None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:21.817322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 110/218] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:49:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:21.822145 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f4f44-f5f1-4f07-905f-ac08653ca59f None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:21.841164 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f245921-2511-457a-a4d7-12f01547914e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:21.841947 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f245921-2511-457a-a4d7-12f01547914e None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:21.842304 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f245921-2511-457a-a4d7-12f01547914e None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:21.842660 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f245921-2511-457a-a4d7-12f01547914e None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:21.843454 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f245921-2511-457a-a4d7-12f01547914e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:21.843767 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f245921-2511-457a-a4d7-12f01547914e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:21.845151 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f245921-2511-457a-a4d7-12f01547914e None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:21.853081 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:21.859041 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 109/219] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:49:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:21.859041 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f245921-2511-457a-a4d7-12f01547914e None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:21.929294 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f4f44-f5f1-4f07-905f-ac08653ca59f None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:21.955005 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:21.962649 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f245921-2511-457a-a4d7-12f01547914e None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:21.964447 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a5e08-d4bc-4a3a-a609-9459a0878ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:21.965324 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a5e08-d4bc-4a3a-a609-9459a0878ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:21.965604 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a5e08-d4bc-4a3a-a609-9459a0878ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:21.965878 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a5e08-d4bc-4a3a-a609-9459a0878ce4 None admin] PATH_INFO: `/v3/users/3e88180080594ae8b15efcea5ddcfd09` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:21.980149 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8a5e08-d4bc-4a3a-a609-9459a0878ce4 None admin] RBAC: Authorizing `identity:get_user(user_id=3e88180080594ae8b15efcea5ddcfd09)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:21.981553 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8a5e08-d4bc-4a3a-a609-9459a0878ce4 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:21.983214 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 111/220] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:49:21 2022] GET /identity/v3/users/3e88180080594ae8b15efcea5ddcfd09 => generated 297 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:21.984011 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a5e08-d4bc-4a3a-a609-9459a0878ce4 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:22.000460 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe15f357-7252-49be-9751-f123bcffc42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:22.001118 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15f357-7252-49be-9751-f123bcffc42a None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:22.001400 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15f357-7252-49be-9751-f123bcffc42a None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:22.001672 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15f357-7252-49be-9751-f123bcffc42a None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:22.002388 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe15f357-7252-49be-9751-f123bcffc42a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:22.003826 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe15f357-7252-49be-9751-f123bcffc42a None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:22.025398 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 110/221] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Jun 7 21:49:21 2022] POST /identity/v3/users => generated 297 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 21:49:22.026693 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690eaa1-5a20-43cf-8149-0ef6ef385b73 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:22.026976 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690eaa1-5a20-43cf-8149-0ef6ef385b73 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:22.027282 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690eaa1-5a20-43cf-8149-0ef6ef385b73 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:22.031388 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 111/222] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:22.031388 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b690eaa1-5a20-43cf-8149-0ef6ef385b73 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:22.094124 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:22.109884 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a5e08-d4bc-4a3a-a609-9459a0878ce4 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:22.150503 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8531acb-2885-4043-a65f-b74753df315a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:22.151425 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8531acb-2885-4043-a65f-b74753df315a None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:22.151530 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b690eaa1-5a20-43cf-8149-0ef6ef385b73 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:22.151729 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8531acb-2885-4043-a65f-b74753df315a None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:22.151978 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8531acb-2885-4043-a65f-b74753df315a None admin] PATH_INFO: `/v3/users/b80029c598ad44d4a8eaf3e161a8bb5b` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:22.153351 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8531acb-2885-4043-a65f-b74753df315a None admin] RBAC: Authorizing `identity:get_user(user_id=b80029c598ad44d4a8eaf3e161a8bb5b)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:22.154967 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8531acb-2885-4043-a65f-b74753df315a None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:22.161297 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 112/223] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:49:21 2022] GET /identity/v3/users/b80029c598ad44d4a8eaf3e161a8bb5b => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:22.161297 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3997b24-4ba5-4057-bea6-9b46918a4990 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:22.161297 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3997b24-4ba5-4057-bea6-9b46918a4990 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:22.161297 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3997b24-4ba5-4057-bea6-9b46918a4990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:22.179920 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-d3997b24-4ba5-4057-bea6-9b46918a4990 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:22.181573 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d3997b24-4ba5-4057-bea6-9b46918a4990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:22.197573 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3654507-d094-453f-b2af-07336c27d4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:22.198602 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3654507-d094-453f-b2af-07336c27d4ed None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:22.198602 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3654507-d094-453f-b2af-07336c27d4ed None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:22.198858 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3654507-d094-453f-b2af-07336c27d4ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:22.224417 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:22.264519 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 113/224] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:49:22.265305 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3997b24-4ba5-4057-bea6-9b46918a4990 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:22.286134 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3654507-d094-453f-b2af-07336c27d4ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:22.288048 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3654507-d094-453f-b2af-07336c27d4ed None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:22.374026 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:22.396824 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3997b24-4ba5-4057-bea6-9b46918a4990 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:22.422542 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 112/225] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:49:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 21:49:22.423286 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3654507-d094-453f-b2af-07336c27d4ed None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:22.445708 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9462aa-c29a-4f1f-be0e-7ccaf7c1e369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:22.446585 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9462aa-c29a-4f1f-be0e-7ccaf7c1e369 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:22.446868 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9462aa-c29a-4f1f-be0e-7ccaf7c1e369 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:22.447141 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9462aa-c29a-4f1f-be0e-7ccaf7c1e369 None admin] PATH_INFO: `/v3/projects/e8ca129f01d149b5aab391353f90139b` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:22.448922 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9462aa-c29a-4f1f-be0e-7ccaf7c1e369 None admin] RBAC: Authorizing `identity:get_project(project_id=e8ca129f01d149b5aab391353f90139b)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:22.451065 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9462aa-c29a-4f1f-be0e-7ccaf7c1e369 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:22.454064 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 114/226] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:49:22 2022] GET /identity/v3/projects/e8ca129f01d149b5aab391353f90139b => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:22.454632 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9462aa-c29a-4f1f-be0e-7ccaf7c1e369 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:22.489095 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:22.558041 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3654507-d094-453f-b2af-07336c27d4ed None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:22.574535 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9462aa-c29a-4f1f-be0e-7ccaf7c1e369 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:22.601074 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156195d5-5130-4a0d-8183-c0c826b8bd81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:22.602270 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156195d5-5130-4a0d-8183-c0c826b8bd81 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:22.602749 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156195d5-5130-4a0d-8183-c0c826b8bd81 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:22.603229 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156195d5-5130-4a0d-8183-c0c826b8bd81 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:22.615005 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156195d5-5130-4a0d-8183-c0c826b8bd81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:22.619164 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fe177e-79ae-4594-aee0-9c75427c858a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:22.620057 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe177e-79ae-4594-aee0-9c75427c858a None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:22.620341 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe177e-79ae-4594-aee0-9c75427c858a None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:22.620617 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe177e-79ae-4594-aee0-9c75427c858a None admin] PATH_INFO: `/v3/projects/e8ca129f01d149b5aab391353f90139b` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:22.621816 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fe177e-79ae-4594-aee0-9c75427c858a None admin] RBAC: Authorizing `identity:get_project(project_id=e8ca129f01d149b5aab391353f90139b)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:22.623268 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fe177e-79ae-4594-aee0-9c75427c858a None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:22.625000 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 115/227] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:49:22 2022] GET /identity/v3/projects/e8ca129f01d149b5aab391353f90139b => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:22.625550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-156195d5-5130-4a0d-8183-c0c826b8bd81 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 21:49:22.625550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:22.625550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:22.625550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:22.625550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:22.625550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:22.625550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:22.625550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:22.625550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:22.625550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:22.625550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:22.625550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:22.625550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:22.625550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:22.625550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:22.625550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:22.626417 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:22.626417 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:22.626417 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:22.626417 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:22.626417 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:22.626417 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:22.626417 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:22.626417 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:22.626417 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:22.626417 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:22.626417 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:22.626417 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:22.626417 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:22.626417 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:22.626417 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:22.626417 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:22.627215 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:22.627215 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:22.627215 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:22.627215 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:22.627215 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:22.627215 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:22.627215 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:22.627215 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:22.627215 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:22.627215 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:22.627215 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 21:49:22.627215 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:49:22.627856 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fe177e-79ae-4594-aee0-9c75427c858a None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:22.630671 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 113/228] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Jun 7 21:49:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:49:22.636194 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156195d5-5130-4a0d-8183-c0c826b8bd81 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:22.659713 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:22.761216 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156195d5-5130-4a0d-8183-c0c826b8bd81 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:22.767651 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fe177e-79ae-4594-aee0-9c75427c858a None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:22.803194 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e85c41-5163-4cc0-94a8-569e6ec4ab3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:22.804332 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e85c41-5163-4cc0-94a8-569e6ec4ab3b None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:22.804819 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e85c41-5163-4cc0-94a8-569e6ec4ab3b None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:22.805347 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e85c41-5163-4cc0-94a8-569e6ec4ab3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:22.806376 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e85c41-5163-4cc0-94a8-569e6ec4ab3b None admin] RBAC: Adding query filter params (role.id=c5431c9deadf47e496264ba5b78eeaf6, scope.project.id=e8ca129f01d149b5aab391353f90139b, user.id=3e88180080594ae8b15efcea5ddcfd09) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:22.806888 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e85c41-5163-4cc0-94a8-569e6ec4ab3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:22.810589 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c7377-ce7d-4b03-88be-d4265cfde8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:22.811447 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c7377-ce7d-4b03-88be-d4265cfde8fc None admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:22.811548 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:22.811737 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c7377-ce7d-4b03-88be-d4265cfde8fc None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:22.812011 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c7377-ce7d-4b03-88be-d4265cfde8fc None admin] PATH_INFO: `/v3/projects/e8ca129f01d149b5aab391353f90139b/users/b80029c598ad44d4a8eaf3e161a8bb5b/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:22.812315 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e85c41-5163-4cc0-94a8-569e6ec4ab3b None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:22.814196 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3c7377-ce7d-4b03-88be-d4265cfde8fc None admin] RBAC: Authorizing `identity:create_grant(project_id=e8ca129f01d149b5aab391353f90139b, user_id=b80029c598ad44d4a8eaf3e161a8bb5b, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:22.816346 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3c7377-ce7d-4b03-88be-d4265cfde8fc None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:22.819441 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 114/229] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:49:22 2022] GET /identity/v3/role_assignments?user.id=3e88180080594ae8b15efcea5ddcfd09&scope.project.id=e8ca129f01d149b5aab391353f90139b&role.id=c5431c9deadf47e496264ba5b78eeaf6 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:22.820468 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e85c41-5163-4cc0-94a8-569e6ec4ab3b None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:22.845117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 116/230] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jun 7 21:49:22 2022] PUT /identity/v3/projects/e8ca129f01d149b5aab391353f90139b/users/b80029c598ad44d4a8eaf3e161a8bb5b/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 21:49:22.986877 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e85c41-5163-4cc0-94a8-569e6ec4ab3b None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:23.027435 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b828b9e5-d450-45dd-bb81-d5b586485d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:23.027647 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828b9e5-d450-45dd-bb81-d5b586485d92 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:23.028066 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828b9e5-d450-45dd-bb81-d5b586485d92 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:23.028583 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828b9e5-d450-45dd-bb81-d5b586485d92 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:23.029893 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b828b9e5-d450-45dd-bb81-d5b586485d92 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:23.030104 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b828b9e5-d450-45dd-bb81-d5b586485d92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:23.031576 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b828b9e5-d450-45dd-bb81-d5b586485d92 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:23.044296 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 115/231] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:49:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:23.050232 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c7377-ce7d-4b03-88be-d4265cfde8fc None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:23.152276 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:23.172148 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c7377-ce7d-4b03-88be-d4265cfde8fc None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:23.215771 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689dfaa0-ceb0-4773-a73e-6de1b2c5cc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:23.216660 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689dfaa0-ceb0-4773-a73e-6de1b2c5cc43 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:23.216945 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689dfaa0-ceb0-4773-a73e-6de1b2c5cc43 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:23.217218 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689dfaa0-ceb0-4773-a73e-6de1b2c5cc43 None admin] PATH_INFO: `/v3/users/b80029c598ad44d4a8eaf3e161a8bb5b` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:23.218417 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689dfaa0-ceb0-4773-a73e-6de1b2c5cc43 None admin] RBAC: Authorizing `identity:get_user(user_id=b80029c598ad44d4a8eaf3e161a8bb5b)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:23.219920 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689dfaa0-ceb0-4773-a73e-6de1b2c5cc43 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:23.222109 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 117/232] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:49:23 2022] GET /identity/v3/users/b80029c598ad44d4a8eaf3e161a8bb5b => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:49:23.227949 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b828b9e5-d450-45dd-bb81-d5b586485d92 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:23.277128 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:23.349336 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b828b9e5-d450-45dd-bb81-d5b586485d92 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:23.384838 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce9673-6939-4486-9149-455db4632107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:23.385755 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce9673-6939-4486-9149-455db4632107 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:23.386045 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce9673-6939-4486-9149-455db4632107 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:23.386325 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce9673-6939-4486-9149-455db4632107 None admin] PATH_INFO: `/v3/projects/e8ca129f01d149b5aab391353f90139b` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:23.387573 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cce9673-6939-4486-9149-455db4632107 None admin] RBAC: Authorizing `identity:get_project(project_id=e8ca129f01d149b5aab391353f90139b)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:23.389054 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cce9673-6939-4486-9149-455db4632107 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:23.395255 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 116/233] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:49:23 2022] GET /identity/v3/projects/e8ca129f01d149b5aab391353f90139b => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:23.400502 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689dfaa0-ceb0-4773-a73e-6de1b2c5cc43 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:23.524852 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689dfaa0-ceb0-4773-a73e-6de1b2c5cc43 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:23.570808 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6421e7b-f19a-4155-8b26-45bef7f6606f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:23.571796 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6421e7b-f19a-4155-8b26-45bef7f6606f None admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:23.572087 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6421e7b-f19a-4155-8b26-45bef7f6606f None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:23.572422 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6421e7b-f19a-4155-8b26-45bef7f6606f None admin] PATH_INFO: `/v3/projects/e8ca129f01d149b5aab391353f90139b/users/b80029c598ad44d4a8eaf3e161a8bb5b/roles/50e04e9d09e045b0bcf8e1782a6e2f23` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:23.574716 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6421e7b-f19a-4155-8b26-45bef7f6606f None admin] RBAC: Authorizing `identity:create_grant(project_id=e8ca129f01d149b5aab391353f90139b, user_id=b80029c598ad44d4a8eaf3e161a8bb5b, role_id=50e04e9d09e045b0bcf8e1782a6e2f23)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:23.576208 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6421e7b-f19a-4155-8b26-45bef7f6606f None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:23.593754 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 118/234] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jun 7 21:49:23 2022] PUT /identity/v3/projects/e8ca129f01d149b5aab391353f90139b/users/b80029c598ad44d4a8eaf3e161a8bb5b/roles/50e04e9d09e045b0bcf8e1782a6e2f23 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 21:49:24.657819 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78cfa37-c557-4d8f-a51d-c71ea8ddbb7f None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:24.658473 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78cfa37-c557-4d8f-a51d-c71ea8ddbb7f None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:24.658473 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78cfa37-c557-4d8f-a51d-c71ea8ddbb7f None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:24.660717 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 117/235] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:24.666601 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd66dc-faa3-4f01-9aca-9a1c38c0433d None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:24.667079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd66dc-faa3-4f01-9aca-9a1c38c0433d None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:24.667245 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd66dc-faa3-4f01-9aca-9a1c38c0433d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:24.693915 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-e9dd66dc-faa3-4f01-9aca-9a1c38c0433d None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:24.695173 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-e9dd66dc-faa3-4f01-9aca-9a1c38c0433d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:24.698560 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:24.798046 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 119/236] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 21:49:24.851559 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78cfa37-c557-4d8f-a51d-c71ea8ddbb7f None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:24.868744 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9a39f-9fe5-425c-a70c-76721b36c353 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:24.869050 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9a39f-9fe5-425c-a70c-76721b36c353 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:24.869294 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9a39f-9fe5-425c-a70c-76721b36c353 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:24.870676 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 120/237] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 21:49:24.885104 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208fe70-f0a4-46ee-8313-66ae04e071da None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:24.885396 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208fe70-f0a4-46ee-8313-66ae04e071da None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:24.885649 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208fe70-f0a4-46ee-8313-66ae04e071da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:24.920425 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-7208fe70-f0a4-46ee-8313-66ae04e071da None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:24.921567 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-7208fe70-f0a4-46ee-8313-66ae04e071da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:24.968792 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78cfa37-c557-4d8f-a51d-c71ea8ddbb7f None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:24.983662 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:25.007219 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e7dd8-3a0e-49c7-b797-a291057d11fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:25.008069 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e7dd8-3a0e-49c7-b797-a291057d11fc None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:25.008357 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e7dd8-3a0e-49c7-b797-a291057d11fc None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:25.008638 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e7dd8-3a0e-49c7-b797-a291057d11fc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:25.009816 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639e7dd8-3a0e-49c7-b797-a291057d11fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:25.011296 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639e7dd8-3a0e-49c7-b797-a291057d11fc None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:25.013533 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 118/238] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:49:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:25.024254 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e7dd8-3a0e-49c7-b797-a291057d11fc None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:25.120985 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 121/239] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 21:49:25.137723 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e7dd8-3a0e-49c7-b797-a291057d11fc None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:25.157100 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7208fe70-f0a4-46ee-8313-66ae04e071da None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:25.176716 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8484de81-239b-4964-9291-c7d813433157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:25.178225 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8484de81-239b-4964-9291-c7d813433157 None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:25.178710 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8484de81-239b-4964-9291-c7d813433157 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:25.179670 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8484de81-239b-4964-9291-c7d813433157 None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:25.180585 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8484de81-239b-4964-9291-c7d813433157 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:25.182272 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8484de81-239b-4964-9291-c7d813433157 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:25.216608 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 119/240] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Jun 7 21:49:25 2022] POST /identity/v3/users => generated 293 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 21:49:25.263882 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:25.316691 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7208fe70-f0a4-46ee-8313-66ae04e071da None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:25.363806 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7909137d-e449-4540-a8e9-41ad677570ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:25.364593 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7909137d-e449-4540-a8e9-41ad677570ff None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:25.364873 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7909137d-e449-4540-a8e9-41ad677570ff None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:25.365146 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7909137d-e449-4540-a8e9-41ad677570ff None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:25.377312 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7909137d-e449-4540-a8e9-41ad677570ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:25.386079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-7909137d-e449-4540-a8e9-41ad677570ff None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 21:49:25.386079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:25.386079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:25.386079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:25.386079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:25.386079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:25.386079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:25.386079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:25.386079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:25.386079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:25.386079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:25.386079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:25.386079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:25.386079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:25.386079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:25.386079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:25.387034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:25.387034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:25.387034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:25.387034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:25.387034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:25.387034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:25.387034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:25.387034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:25.387034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:25.387034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:25.387034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:25.387034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:25.387034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:25.387034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:25.387034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:25.387034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:25.387889 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:25.387889 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:25.387889 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:25.387889 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:25.387889 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:25.387889 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:25.387889 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:25.387889 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:25.387889 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:25.387889 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:25.387889 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 21:49:25.387889 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:49:25.387889 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 122/241] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:49:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:49:25.393216 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8484de81-239b-4964-9291-c7d813433157 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:25.432832 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:25.507835 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8484de81-239b-4964-9291-c7d813433157 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:25.545487 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711668c-2419-4096-b000-c3098e92ab83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:25.546163 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711668c-2419-4096-b000-c3098e92ab83 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:25.546444 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711668c-2419-4096-b000-c3098e92ab83 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:25.546719 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711668c-2419-4096-b000-c3098e92ab83 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:25.547511 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6711668c-2419-4096-b000-c3098e92ab83 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:25.547817 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6711668c-2419-4096-b000-c3098e92ab83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:25.549196 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6711668c-2419-4096-b000-c3098e92ab83 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:25.558701 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 120/242] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jun 7 21:49:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:49:25.563738 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7909137d-e449-4540-a8e9-41ad677570ff None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:25.687825 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246bc204-5af5-4b62-9d0e-1df0c5494238 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:25.688304 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246bc204-5af5-4b62-9d0e-1df0c5494238 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:25.688304 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246bc204-5af5-4b62-9d0e-1df0c5494238 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:25.689796 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 121/243] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 21:49:25.696530 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388dd98-d3eb-4e6f-bbe7-b053c2ba24ee None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:25.696819 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388dd98-d3eb-4e6f-bbe7-b053c2ba24ee None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:25.697102 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388dd98-d3eb-4e6f-bbe7-b053c2ba24ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:25.718279 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7909137d-e449-4540-a8e9-41ad677570ff None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:25.724793 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-9388dd98-d3eb-4e6f-bbe7-b053c2ba24ee None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:25.731718 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-9388dd98-d3eb-4e6f-bbe7-b053c2ba24ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:25.770812 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:25.784747 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016102fb-acdc-4c4a-b9ef-dfd6fda38b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:25.785751 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016102fb-acdc-4c4a-b9ef-dfd6fda38b22 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:25.786033 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016102fb-acdc-4c4a-b9ef-dfd6fda38b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:25.786302 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016102fb-acdc-4c4a-b9ef-dfd6fda38b22 None admin] PATH_INFO: `/v3/users/50d42f665bc848b7b2a98aa9cad7aef2` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:25.806220 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016102fb-acdc-4c4a-b9ef-dfd6fda38b22 None admin] RBAC: Authorizing `identity:get_user(user_id=50d42f665bc848b7b2a98aa9cad7aef2)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:25.807735 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016102fb-acdc-4c4a-b9ef-dfd6fda38b22 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:25.809623 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 123/244] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:49:25 2022] GET /identity/v3/users/50d42f665bc848b7b2a98aa9cad7aef2 => generated 297 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:25.820356 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016102fb-acdc-4c4a-b9ef-dfd6fda38b22 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:25.857338 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 122/245] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 21:49:25.886458 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388dd98-d3eb-4e6f-bbe7-b053c2ba24ee None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:25.917271 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:25.962927 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016102fb-acdc-4c4a-b9ef-dfd6fda38b22 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:26.005507 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0220cd58-9a3b-4986-ab9e-bf2e7a42e0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:26.006171 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0220cd58-9a3b-4986-ab9e-bf2e7a42e0d1 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:26.006453 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0220cd58-9a3b-4986-ab9e-bf2e7a42e0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:26.006727 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0220cd58-9a3b-4986-ab9e-bf2e7a42e0d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:26.026986 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388dd98-d3eb-4e6f-bbe7-b053c2ba24ee None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:26.093840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:26.100191 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fdba4b-cc50-41b6-8013-bc89b21077d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:26.101002 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fdba4b-cc50-41b6-8013-bc89b21077d1 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:26.101287 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fdba4b-cc50-41b6-8013-bc89b21077d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:26.101565 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fdba4b-cc50-41b6-8013-bc89b21077d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:26.119796 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0220cd58-9a3b-4986-ab9e-bf2e7a42e0d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:26.121279 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0220cd58-9a3b-4986-ab9e-bf2e7a42e0d1 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:26.127043 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fdba4b-cc50-41b6-8013-bc89b21077d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:26.134845 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-c6fdba4b-cc50-41b6-8013-bc89b21077d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:49:26.134845 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:26.134845 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:26.134845 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:26.134845 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:26.134845 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:26.134845 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:26.134845 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:26.134845 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:26.134845 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:26.134845 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:26.134845 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:26.134845 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:26.134845 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:26.134845 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:26.134845 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:26.136030 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:26.136030 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:26.136030 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:26.136030 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:26.136030 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:26.136030 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:26.136030 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:26.136030 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:26.136030 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:26.136030 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:26.136030 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:26.136030 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:26.136030 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:26.136030 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:26.136030 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:26.136030 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:26.137098 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:26.137098 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:26.137098 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:26.137098 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:26.137098 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:26.137098 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:26.137098 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:26.137098 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:26.137098 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:26.137098 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:26.137098 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:49:26.137098 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:49:26.137098 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 123/246] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jun 7 21:49:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:49:26.137743 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af7570-fc00-46c0-8b66-86d0808d7c64 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:26.138077 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af7570-fc00-46c0-8b66-86d0808d7c64 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:26.138278 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af7570-fc00-46c0-8b66-86d0808d7c64 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:26.147361 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 124/247] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:26.147361 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7af7570-fc00-46c0-8b66-86d0808d7c64 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:26.232893 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:26.267360 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7af7570-fc00-46c0-8b66-86d0808d7c64 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:26.285496 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 124/248] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:49:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 21:49:26.286660 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f944a2-5e96-4ce9-b3a0-2269a37928bb None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:26.286952 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f944a2-5e96-4ce9-b3a0-2269a37928bb None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:26.287227 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f944a2-5e96-4ce9-b3a0-2269a37928bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:26.331719 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-37f944a2-5e96-4ce9-b3a0-2269a37928bb None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:26.332977 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-37f944a2-5e96-4ce9-b3a0-2269a37928bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:26.349172 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fba5668-6f09-445d-909b-5d645dfe477a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:26.349898 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba5668-6f09-445d-909b-5d645dfe477a None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:26.350182 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba5668-6f09-445d-909b-5d645dfe477a None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:26.350542 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba5668-6f09-445d-909b-5d645dfe477a None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:26.351805 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fba5668-6f09-445d-909b-5d645dfe477a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:26.352281 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fba5668-6f09-445d-909b-5d645dfe477a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:26.356677 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fba5668-6f09-445d-909b-5d645dfe477a None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:26.386060 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 125/249] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:49:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:26.386717 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fba5668-6f09-445d-909b-5d645dfe477a None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:26.510963 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 125/250] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:49:26.511763 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f944a2-5e96-4ce9-b3a0-2269a37928bb None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:26.525993 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fba5668-6f09-445d-909b-5d645dfe477a None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:26.561095 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:26.579586 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecc44df-584d-49e5-b154-7b404788a26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:26.580470 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc44df-584d-49e5-b154-7b404788a26a None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:26.580757 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc44df-584d-49e5-b154-7b404788a26a None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:26.581036 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc44df-584d-49e5-b154-7b404788a26a None admin] PATH_INFO: `/v3/projects/e8ca129f01d149b5aab391353f90139b` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:26.582279 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecc44df-584d-49e5-b154-7b404788a26a None admin] RBAC: Authorizing `identity:get_project(project_id=e8ca129f01d149b5aab391353f90139b)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:26.583923 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecc44df-584d-49e5-b154-7b404788a26a None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:26.586184 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 126/251] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:49:26 2022] GET /identity/v3/projects/e8ca129f01d149b5aab391353f90139b => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:26.586997 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecc44df-584d-49e5-b154-7b404788a26a None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:26.614779 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:26.664080 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f944a2-5e96-4ce9-b3a0-2269a37928bb None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:26.704781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8c7256-64df-4eb3-9731-53d3fd4b2494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:26.705655 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c7256-64df-4eb3-9731-53d3fd4b2494 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:26.705936 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c7256-64df-4eb3-9731-53d3fd4b2494 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:26.706210 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c7256-64df-4eb3-9731-53d3fd4b2494 None admin] PATH_INFO: `/v3/users/b80029c598ad44d4a8eaf3e161a8bb5b` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:26.707468 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8c7256-64df-4eb3-9731-53d3fd4b2494 None admin] RBAC: Authorizing `identity:get_user(user_id=b80029c598ad44d4a8eaf3e161a8bb5b)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:26.708930 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8c7256-64df-4eb3-9731-53d3fd4b2494 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:26.710579 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 126/252] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:49:26 2022] GET /identity/v3/users/b80029c598ad44d4a8eaf3e161a8bb5b => generated 283 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:26.711472 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8c7256-64df-4eb3-9731-53d3fd4b2494 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:26.746122 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecc44df-584d-49e5-b154-7b404788a26a None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:26.802538 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f35c87e-3821-4a08-ace4-f7c4724d9a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:26.804505 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f35c87e-3821-4a08-ace4-f7c4724d9a70 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:26.805034 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f35c87e-3821-4a08-ace4-f7c4724d9a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:26.805812 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f35c87e-3821-4a08-ace4-f7c4724d9a70 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:26.808014 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:26.823960 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f35c87e-3821-4a08-ace4-f7c4724d9a70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:26.825268 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8c7256-64df-4eb3-9731-53d3fd4b2494 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:26.834198 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-4f35c87e-3821-4a08-ace4-f7c4724d9a70 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 21:49:26.834198 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:26.834198 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:26.834198 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:26.834198 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:26.834198 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:26.834198 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:26.834198 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:26.834198 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:26.834198 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:26.834198 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:26.834198 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:26.834198 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:26.834198 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:26.834198 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:26.834198 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:26.835121 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:26.835121 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:26.835121 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:26.835121 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:26.835121 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:26.835121 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:26.835121 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:26.835121 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:26.835121 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:26.835121 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:26.835121 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:26.835121 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:26.835121 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:26.835121 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:26.835121 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:26.835121 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:26.836589 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:26.836589 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:26.836589 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:26.836589 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:26.836589 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:26.836589 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:26.836589 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:26.836589 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:26.836589 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:26.836589 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:26.836589 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 21:49:26.836589 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:49:26.838156 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 127/253] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:49:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:49:26.839476 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2bb4c-c278-485e-b963-faddc9b19516 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:26.839765 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2bb4c-c278-485e-b963-faddc9b19516 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:26.840044 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2bb4c-c278-485e-b963-faddc9b19516 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:26.841340 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 128/254] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:26.845594 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d2bb4c-c278-485e-b963-faddc9b19516 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:26.874264 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9bfcab-29dc-4847-be22-b9d4d3091c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:26.875262 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9bfcab-29dc-4847-be22-b9d4d3091c28 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:26.875767 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9bfcab-29dc-4847-be22-b9d4d3091c28 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:26.876256 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9bfcab-29dc-4847-be22-b9d4d3091c28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:26.877272 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9bfcab-29dc-4847-be22-b9d4d3091c28 None admin] RBAC: Adding query filter params (role.id=30b0c2ba85ff481eb256122140e4163a, scope.project.id=e8ca129f01d149b5aab391353f90139b, user.id=50d42f665bc848b7b2a98aa9cad7aef2) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:26.877790 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9bfcab-29dc-4847-be22-b9d4d3091c28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:26.879485 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9bfcab-29dc-4847-be22-b9d4d3091c28 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:26.885938 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 127/255] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:49:26 2022] GET /identity/v3/role_assignments?user.id=50d42f665bc848b7b2a98aa9cad7aef2&scope.project.id=e8ca129f01d149b5aab391353f90139b&role.id=30b0c2ba85ff481eb256122140e4163a => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:26.887068 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9bfcab-29dc-4847-be22-b9d4d3091c28 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:26.971983 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:27.028749 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d2bb4c-c278-485e-b963-faddc9b19516 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:27.060493 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9bfcab-29dc-4847-be22-b9d4d3091c28 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:27.094083 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4874e880-6267-460b-a13d-ee8644f117d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:27.094751 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4874e880-6267-460b-a13d-ee8644f117d4 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:27.095075 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4874e880-6267-460b-a13d-ee8644f117d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:27.095315 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4874e880-6267-460b-a13d-ee8644f117d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:27.100434 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb662b8b-f1fa-4d30-aabd-0ec48ba52a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:27.101399 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb662b8b-f1fa-4d30-aabd-0ec48ba52a89 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:27.101900 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb662b8b-f1fa-4d30-aabd-0ec48ba52a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:27.102374 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb662b8b-f1fa-4d30-aabd-0ec48ba52a89 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:27.103943 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb662b8b-f1fa-4d30-aabd-0ec48ba52a89 None admin] RBAC: Adding query filter params (name=member) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:27.104476 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb662b8b-f1fa-4d30-aabd-0ec48ba52a89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:27.106167 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb662b8b-f1fa-4d30-aabd-0ec48ba52a89 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:27.115910 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 128/256] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jun 7 21:49:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:27.117444 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e1a16-2f33-467c-bdbc-96f6042129f0 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:27.117929 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e1a16-2f33-467c-bdbc-96f6042129f0 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:27.118513 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e1a16-2f33-467c-bdbc-96f6042129f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:27.137527 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-7c8e1a16-2f33-467c-bdbc-96f6042129f0 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:27.139221 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-7c8e1a16-2f33-467c-bdbc-96f6042129f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:27.148326 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:27.205497 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4874e880-6267-460b-a13d-ee8644f117d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:27.207139 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4874e880-6267-460b-a13d-ee8644f117d4 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:27.217843 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 129/257] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:49:27.218946 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e1a16-2f33-467c-bdbc-96f6042129f0 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:27.329244 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e1a16-2f33-467c-bdbc-96f6042129f0 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:27.364420 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:27.370397 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eb28ee-01c6-46b7-8aa3-686a470f08e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:27.371575 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb28ee-01c6-46b7-8aa3-686a470f08e6 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:27.372077 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb28ee-01c6-46b7-8aa3-686a470f08e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:27.372564 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb28ee-01c6-46b7-8aa3-686a470f08e6 None admin] PATH_INFO: `/v3/users/3e88180080594ae8b15efcea5ddcfd09` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:27.374033 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eb28ee-01c6-46b7-8aa3-686a470f08e6 None admin] RBAC: Authorizing `identity:get_user(user_id=3e88180080594ae8b15efcea5ddcfd09)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:27.375729 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eb28ee-01c6-46b7-8aa3-686a470f08e6 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:27.377599 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 130/258] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:49:27 2022] GET /identity/v3/users/3e88180080594ae8b15efcea5ddcfd09 => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:27.378552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eb28ee-01c6-46b7-8aa3-686a470f08e6 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:27.391143 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:27.429267 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 129/259] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:49:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 21:49:27.429908 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4874e880-6267-460b-a13d-ee8644f117d4 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:27.490538 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eb28ee-01c6-46b7-8aa3-686a470f08e6 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:27.527569 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafbd1fa-37af-4cac-9f8c-3f8f4d837b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:27.528838 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafbd1fa-37af-4cac-9f8c-3f8f4d837b26 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:27.529400 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafbd1fa-37af-4cac-9f8c-3f8f4d837b26 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:27.529867 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafbd1fa-37af-4cac-9f8c-3f8f4d837b26 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:27.541521 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafbd1fa-37af-4cac-9f8c-3f8f4d837b26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:27.552378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-fafbd1fa-37af-4cac-9f8c-3f8f4d837b26 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 21:49:27.552378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:27.552378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:27.552378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:27.552378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:27.552378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:27.552378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:27.552378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:27.552378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:27.552378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:27.552378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:27.552378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:27.552378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:27.552378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:27.552378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:27.552378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:27.553291 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:27.553291 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:27.553291 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:27.553291 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:27.553291 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:27.553291 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:27.553291 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:27.553291 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:27.553291 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:27.553291 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:27.553291 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:27.553291 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:27.553291 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:27.553291 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:27.553291 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:27.553291 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:27.554157 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:27.554157 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:27.554157 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:27.554157 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:27.554157 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:27.554157 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:27.554157 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:27.554157 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:27.554157 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:27.554157 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:27.554157 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 21:49:27.554157 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:49:27.556598 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 131/260] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Jun 7 21:49:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:49:27.557678 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafbd1fa-37af-4cac-9f8c-3f8f4d837b26 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:27.635762 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:27.669320 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafbd1fa-37af-4cac-9f8c-3f8f4d837b26 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:27.706577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bfac07-b494-4984-831b-0b663bb49d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:27.707739 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bfac07-b494-4984-831b-0b663bb49d94 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:27.708159 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bfac07-b494-4984-831b-0b663bb49d94 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:27.708642 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bfac07-b494-4984-831b-0b663bb49d94 None admin] PATH_INFO: `/v3/projects/e8ca129f01d149b5aab391353f90139b` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:27.709920 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bfac07-b494-4984-831b-0b663bb49d94 None admin] RBAC: Authorizing `identity:get_project(project_id=e8ca129f01d149b5aab391353f90139b)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:27.711483 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bfac07-b494-4984-831b-0b663bb49d94 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:27.713607 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 132/261] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:49:27 2022] GET /identity/v3/projects/e8ca129f01d149b5aab391353f90139b => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:27.714279 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bfac07-b494-4984-831b-0b663bb49d94 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:27.790221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4874e880-6267-460b-a13d-ee8644f117d4 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:27.838182 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bfac07-b494-4984-831b-0b663bb49d94 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:27.880001 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5402172-1d5e-46c3-b73b-68ff6fd25b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:27.881710 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5402172-1d5e-46c3-b73b-68ff6fd25b3d None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:27.883459 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5402172-1d5e-46c3-b73b-68ff6fd25b3d None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:27.883943 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5402172-1d5e-46c3-b73b-68ff6fd25b3d None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:27.884903 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5402172-1d5e-46c3-b73b-68ff6fd25b3d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:27.886362 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5402172-1d5e-46c3-b73b-68ff6fd25b3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:27.889208 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5402172-1d5e-46c3-b73b-68ff6fd25b3d None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:27.904270 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd433cd2-ddf0-479d-ac5d-617d9072cc8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:27.905492 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd433cd2-ddf0-479d-ac5d-617d9072cc8f None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:27.905969 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd433cd2-ddf0-479d-ac5d-617d9072cc8f None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:27.906469 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd433cd2-ddf0-479d-ac5d-617d9072cc8f None admin] PATH_INFO: `/v3/projects/e8ca129f01d149b5aab391353f90139b` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:27.909760 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd433cd2-ddf0-479d-ac5d-617d9072cc8f None admin] RBAC: Authorizing `identity:get_project(project_id=e8ca129f01d149b5aab391353f90139b)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:27.911682 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd433cd2-ddf0-479d-ac5d-617d9072cc8f None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:27.912108 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 133/262] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:49:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:27.913239 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5402172-1d5e-46c3-b73b-68ff6fd25b3d None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:27.918167 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 130/263] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:49:27 2022] GET /identity/v3/projects/e8ca129f01d149b5aab391353f90139b => generated 302 bytes in 489 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:27.919257 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd433cd2-ddf0-479d-ac5d-617d9072cc8f None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:27.919668 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:28.016155 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:28.095512 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5402172-1d5e-46c3-b73b-68ff6fd25b3d None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:28.142401 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd433cd2-ddf0-479d-ac5d-617d9072cc8f None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:28.144890 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91150283-ed29-43cb-acac-771c86dbb9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:28.145556 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91150283-ed29-43cb-acac-771c86dbb9b2 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:28.145879 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91150283-ed29-43cb-acac-771c86dbb9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:28.146235 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91150283-ed29-43cb-acac-771c86dbb9b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:28.147081 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91150283-ed29-43cb-acac-771c86dbb9b2 None admin] RBAC: Adding query filter params (role.id=6b483758dbd44eb4a78df2ce4603d601, scope.project.id=e8ca129f01d149b5aab391353f90139b, user.id=b80029c598ad44d4a8eaf3e161a8bb5b) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:28.147415 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91150283-ed29-43cb-acac-771c86dbb9b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:28.148874 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91150283-ed29-43cb-acac-771c86dbb9b2 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:28.162705 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 134/264] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:49:27 2022] GET /identity/v3/role_assignments?user.id=b80029c598ad44d4a8eaf3e161a8bb5b&scope.project.id=e8ca129f01d149b5aab391353f90139b&role.id=6b483758dbd44eb4a78df2ce4603d601 => generated 608 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:28.164506 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91150283-ed29-43cb-acac-771c86dbb9b2 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:28.235245 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83daa3cb-e502-477f-8bd5-dab008b2ab00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:28.236226 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83daa3cb-e502-477f-8bd5-dab008b2ab00 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:28.236506 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83daa3cb-e502-477f-8bd5-dab008b2ab00 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:28.236778 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83daa3cb-e502-477f-8bd5-dab008b2ab00 None admin] PATH_INFO: `/v3/users/b80029c598ad44d4a8eaf3e161a8bb5b` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:28.240488 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83daa3cb-e502-477f-8bd5-dab008b2ab00 None admin] RBAC: Authorizing `identity:get_user(user_id=b80029c598ad44d4a8eaf3e161a8bb5b)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:28.242714 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83daa3cb-e502-477f-8bd5-dab008b2ab00 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:28.250853 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 131/265] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:49:27 2022] GET /identity/v3/users/b80029c598ad44d4a8eaf3e161a8bb5b => generated 283 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:28.252297 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83daa3cb-e502-477f-8bd5-dab008b2ab00 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:28.314083 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:28.373406 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83daa3cb-e502-477f-8bd5-dab008b2ab00 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:28.385135 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:28.393874 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91150283-ed29-43cb-acac-771c86dbb9b2 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:28.414961 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b737b76f-fae6-4737-94ea-2753f085125f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:28.415656 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737b76f-fae6-4737-94ea-2753f085125f None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:28.415941 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737b76f-fae6-4737-94ea-2753f085125f None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:28.416220 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737b76f-fae6-4737-94ea-2753f085125f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:28.464218 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30a72d9-019f-4f5f-8171-b2bcc97bff9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:28.465515 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a72d9-019f-4f5f-8171-b2bcc97bff9a None admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:28.466031 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a72d9-019f-4f5f-8171-b2bcc97bff9a None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:28.466604 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a72d9-019f-4f5f-8171-b2bcc97bff9a None admin] PATH_INFO: `/v3/projects/e8ca129f01d149b5aab391353f90139b/users/3e88180080594ae8b15efcea5ddcfd09/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:28.469314 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30a72d9-019f-4f5f-8171-b2bcc97bff9a None admin] RBAC: Authorizing `identity:create_grant(project_id=e8ca129f01d149b5aab391353f90139b, user_id=3e88180080594ae8b15efcea5ddcfd09, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:28.471244 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30a72d9-019f-4f5f-8171-b2bcc97bff9a None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:28.498009 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b737b76f-fae6-4737-94ea-2753f085125f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:28.499571 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b737b76f-fae6-4737-94ea-2753f085125f None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:28.501104 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 135/266] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jun 7 21:49:28 2022] PUT /identity/v3/projects/e8ca129f01d149b5aab391353f90139b/users/3e88180080594ae8b15efcea5ddcfd09/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 21:49:28.602623 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1070c4a0-c7ea-4917-8fd9-bc794d9e0698 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:28.603282 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1070c4a0-c7ea-4917-8fd9-bc794d9e0698 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:28.603804 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1070c4a0-c7ea-4917-8fd9-bc794d9e0698 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:28.605887 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 136/267] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:28.611221 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862b272-7b79-4409-a50a-93b975386437 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:28.611797 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862b272-7b79-4409-a50a-93b975386437 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:28.612309 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862b272-7b79-4409-a50a-93b975386437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:28.619318 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 132/268] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:49:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 21:49:28.620758 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b737b76f-fae6-4737-94ea-2753f085125f None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:28.636340 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:28.651451 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-3862b272-7b79-4409-a50a-93b975386437 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:28.651451 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-3862b272-7b79-4409-a50a-93b975386437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:28.733461 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 137/269] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:49:28.734519 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b737b76f-fae6-4737-94ea-2753f085125f None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:28.734830 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6113d-436d-4766-a984-c06352d297e9 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:28.735120 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6113d-436d-4766-a984-c06352d297e9 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:28.739169 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6113d-436d-4766-a984-c06352d297e9 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:28.741037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 138/270] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:28.747143 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c7291-b9d8-49c1-9328-646c80c28c88 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:28.747456 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c7291-b9d8-49c1-9328-646c80c28c88 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:28.747840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c7291-b9d8-49c1-9328-646c80c28c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:28.775850 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0061304d-914e-4a88-9b77-45b126a4ee4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:28.776823 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0061304d-914e-4a88-9b77-45b126a4ee4e None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:28.777168 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0061304d-914e-4a88-9b77-45b126a4ee4e None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:28.777503 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0061304d-914e-4a88-9b77-45b126a4ee4e None admin] PATH_INFO: `/v3/projects/e8ca129f01d149b5aab391353f90139b` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:28.778806 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0061304d-914e-4a88-9b77-45b126a4ee4e None admin] RBAC: Authorizing `identity:get_project(project_id=e8ca129f01d149b5aab391353f90139b)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:28.780477 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0061304d-914e-4a88-9b77-45b126a4ee4e None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:28.784459 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-dd3c7291-b9d8-49c1-9328-646c80c28c88 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:28.786252 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-dd3c7291-b9d8-49c1-9328-646c80c28c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:28.787345 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 133/271] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:49:28 2022] GET /identity/v3/projects/e8ca129f01d149b5aab391353f90139b => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:49:28.788743 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0061304d-914e-4a88-9b77-45b126a4ee4e None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:28.861782 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:28.890291 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 139/272] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 21:49:28.891817 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3c7291-b9d8-49c1-9328-646c80c28c88 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:28.893126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:28.937508 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0061304d-914e-4a88-9b77-45b126a4ee4e None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:28.975928 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00b9310-46c8-435e-88e1-ae0c3a1ec123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:28.976591 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00b9310-46c8-435e-88e1-ae0c3a1ec123 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:28.976874 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00b9310-46c8-435e-88e1-ae0c3a1ec123 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:28.977157 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00b9310-46c8-435e-88e1-ae0c3a1ec123 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:28.977991 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00b9310-46c8-435e-88e1-ae0c3a1ec123 None admin] RBAC: Adding query filter params (role.id=50e04e9d09e045b0bcf8e1782a6e2f23, scope.project.id=e8ca129f01d149b5aab391353f90139b, user.id=b80029c598ad44d4a8eaf3e161a8bb5b) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:28.978301 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00b9310-46c8-435e-88e1-ae0c3a1ec123 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:28.979982 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00b9310-46c8-435e-88e1-ae0c3a1ec123 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:28.986238 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 134/273] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:49:28 2022] GET /identity/v3/role_assignments?user.id=b80029c598ad44d4a8eaf3e161a8bb5b&scope.project.id=e8ca129f01d149b5aab391353f90139b&role.id=50e04e9d09e045b0bcf8e1782a6e2f23 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:28.987026 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00b9310-46c8-435e-88e1-ae0c3a1ec123 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:29.069818 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3c7291-b9d8-49c1-9328-646c80c28c88 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:29.126967 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e052462-a52c-437e-a6f3-ddf62070ab9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:29.127810 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e052462-a52c-437e-a6f3-ddf62070ab9b None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:29.128091 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e052462-a52c-437e-a6f3-ddf62070ab9b None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:29.128364 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e052462-a52c-437e-a6f3-ddf62070ab9b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:29.146280 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00b9310-46c8-435e-88e1-ae0c3a1ec123 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:29.152199 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e052462-a52c-437e-a6f3-ddf62070ab9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:29.166995 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-7e052462-a52c-437e-a6f3-ddf62070ab9b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 21:49:29.166995 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:29.166995 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:29.166995 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:29.166995 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:29.166995 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:29.166995 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:29.166995 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:29.166995 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:29.166995 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:29.166995 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:29.166995 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:29.166995 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:29.166995 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:29.166995 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:29.166995 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:29.168006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:29.168006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:29.168006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:29.168006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:29.168006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:29.168006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:29.168006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:29.168006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:29.168006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:29.168006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:29.168006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:29.168006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:29.168006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:29.168006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:29.168006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:29.168006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:29.168811 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:29.168811 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:29.168811 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:29.168811 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:29.168811 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:29.168811 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:29.168811 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:29.168811 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:29.168811 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:29.168811 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:29.168811 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 21:49:29.168811 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:49:29.168811 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 140/274] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:49:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:49:29.175545 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e052462-a52c-437e-a6f3-ddf62070ab9b None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:29.224337 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee5723b-7936-4993-ae6c-c48181c1d286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:29.225336 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5723b-7936-4993-ae6c-c48181c1d286 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:29.225623 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5723b-7936-4993-ae6c-c48181c1d286 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:29.225894 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5723b-7936-4993-ae6c-c48181c1d286 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:29.227783 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee5723b-7936-4993-ae6c-c48181c1d286 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:29.229508 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee5723b-7936-4993-ae6c-c48181c1d286 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:29.232729 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 135/275] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:49:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:29.237669 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee5723b-7936-4993-ae6c-c48181c1d286 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:29.252811 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:29.256201 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:29.331262 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e052462-a52c-437e-a6f3-ddf62070ab9b None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:29.366126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee5723b-7936-4993-ae6c-c48181c1d286 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:29.376601 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b1fe8-2ee8-4938-b725-df28f60dcda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:29.377740 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b1fe8-2ee8-4938-b725-df28f60dcda2 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:29.378023 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b1fe8-2ee8-4938-b725-df28f60dcda2 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:29.378299 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b1fe8-2ee8-4938-b725-df28f60dcda2 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:29.380033 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339b1fe8-2ee8-4938-b725-df28f60dcda2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:29.380692 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339b1fe8-2ee8-4938-b725-df28f60dcda2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:29.382358 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339b1fe8-2ee8-4938-b725-df28f60dcda2 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:29.393989 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 141/276] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jun 7 21:49:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:29.399669 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b1fe8-2ee8-4938-b725-df28f60dcda2 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:29.421423 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a52a24-3157-4052-ae0c-f2374aefab30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:29.422148 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a52a24-3157-4052-ae0c-f2374aefab30 None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:29.422430 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a52a24-3157-4052-ae0c-f2374aefab30 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:29.422702 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a52a24-3157-4052-ae0c-f2374aefab30 None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:29.423434 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a52a24-3157-4052-ae0c-f2374aefab30 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:29.424881 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a52a24-3157-4052-ae0c-f2374aefab30 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:29.457100 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 136/277] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Jun 7 21:49:29 2022] POST /identity/v3/users => generated 293 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 21:49:29.544381 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b1fe8-2ee8-4938-b725-df28f60dcda2 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:29.553887 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:29.583842 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f9e23-20ba-4196-a3d3-925abc9bba69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:29.584740 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f9e23-20ba-4196-a3d3-925abc9bba69 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:29.585023 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f9e23-20ba-4196-a3d3-925abc9bba69 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:29.585301 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f9e23-20ba-4196-a3d3-925abc9bba69 None admin] PATH_INFO: `/v3/users/316a5a2476284a778d902dca87bae5c6` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:29.601028 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619f9e23-20ba-4196-a3d3-925abc9bba69 None admin] RBAC: Authorizing `identity:get_user(user_id=316a5a2476284a778d902dca87bae5c6)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:29.602554 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619f9e23-20ba-4196-a3d3-925abc9bba69 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:29.604797 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 142/278] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:49:29 2022] GET /identity/v3/users/316a5a2476284a778d902dca87bae5c6 => generated 293 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:49:29.610500 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a52a24-3157-4052-ae0c-f2374aefab30 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:29.680165 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:29.725573 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a52a24-3157-4052-ae0c-f2374aefab30 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:29.763463 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e5507-8ff6-4d26-ad37-35bf46ce9a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:29.766206 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e5507-8ff6-4d26-ad37-35bf46ce9a7f None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:29.766206 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e5507-8ff6-4d26-ad37-35bf46ce9a7f None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:29.766206 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e5507-8ff6-4d26-ad37-35bf46ce9a7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:29.766206 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80e5507-8ff6-4d26-ad37-35bf46ce9a7f None admin] RBAC: Adding query filter params (role.id=c5431c9deadf47e496264ba5b78eeaf6, user.id=316a5a2476284a778d902dca87bae5c6, scope.system=all) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:29.766206 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80e5507-8ff6-4d26-ad37-35bf46ce9a7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:29.768532 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80e5507-8ff6-4d26-ad37-35bf46ce9a7f None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:29.773882 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 137/279] 10.222.0.13 () {56 vars in 1308 bytes} [Tue Jun 7 21:49:29 2022] GET /identity/v3/role_assignments?user.id=316a5a2476284a778d902dca87bae5c6&scope.system=all&role.id=c5431c9deadf47e496264ba5b78eeaf6 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:29.824879 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e750e5-9bae-48b7-a2d1-aabf6e646652 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:29.825162 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e750e5-9bae-48b7-a2d1-aabf6e646652 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:29.825362 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e750e5-9bae-48b7-a2d1-aabf6e646652 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:29.826863 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 143/280] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:29.833605 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4148c-5986-4e67-9b6e-fb5a78a4386d None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:29.834156 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4148c-5986-4e67-9b6e-fb5a78a4386d None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:29.834444 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4148c-5986-4e67-9b6e-fb5a78a4386d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:29.854544 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-0ff4148c-5986-4e67-9b6e-fb5a78a4386d None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:29.855874 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-0ff4148c-5986-4e67-9b6e-fb5a78a4386d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:29.947009 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 138/281] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:49:29.971215 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e750e5-9bae-48b7-a2d1-aabf6e646652 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:30.085062 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e750e5-9bae-48b7-a2d1-aabf6e646652 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:30.132984 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40721311-3c0f-473d-b0b0-e7fcc9ddc1b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:30.134207 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40721311-3c0f-473d-b0b0-e7fcc9ddc1b4 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:30.134685 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40721311-3c0f-473d-b0b0-e7fcc9ddc1b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:30.135201 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40721311-3c0f-473d-b0b0-e7fcc9ddc1b4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:30.147134 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40721311-3c0f-473d-b0b0-e7fcc9ddc1b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:30.158484 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-40721311-3c0f-473d-b0b0-e7fcc9ddc1b4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 21:49:30.158484 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:30.158484 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:30.158484 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:30.158484 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:30.158484 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:30.158484 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:30.158484 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:30.158484 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:30.158484 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:30.158484 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:30.158484 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:30.158484 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:30.158484 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:30.158484 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:30.158484 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:30.159392 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:30.159392 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:30.159392 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:30.159392 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:30.159392 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:30.159392 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:30.159392 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:30.159392 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:30.159392 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:30.159392 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:30.159392 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:30.159392 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:30.159392 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:30.159392 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:30.159392 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:30.159392 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:30.160376 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:30.160376 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:30.160376 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:30.160376 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:30.160376 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:30.160376 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:30.160376 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:30.160376 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:30.160376 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:30.160376 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:30.160376 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 21:49:30.160376 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:49:30.162995 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 144/282] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:49:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:49:30.171351 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff4148c-5986-4e67-9b6e-fb5a78a4386d None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:30.280404 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:30.338146 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff4148c-5986-4e67-9b6e-fb5a78a4386d None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:30.388317 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2031db8-4e69-4e6b-996c-f8a5fad03b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:30.388996 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2031db8-4e69-4e6b-996c-f8a5fad03b4e None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:30.389278 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2031db8-4e69-4e6b-996c-f8a5fad03b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:30.389556 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2031db8-4e69-4e6b-996c-f8a5fad03b4e None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:30.390314 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2031db8-4e69-4e6b-996c-f8a5fad03b4e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:30.390623 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2031db8-4e69-4e6b-996c-f8a5fad03b4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:30.392128 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2031db8-4e69-4e6b-996c-f8a5fad03b4e None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:30.407497 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 139/283] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jun 7 21:49:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:30.413385 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40721311-3c0f-473d-b0b0-e7fcc9ddc1b4 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:30.457771 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:30.562333 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40721311-3c0f-473d-b0b0-e7fcc9ddc1b4 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:30.615537 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c1064a-218a-4628-bbea-52eb843e8a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:30.616331 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c1064a-218a-4628-bbea-52eb843e8a72 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:30.616610 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c1064a-218a-4628-bbea-52eb843e8a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:30.616884 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c1064a-218a-4628-bbea-52eb843e8a72 None admin] PATH_INFO: `/v3/users/50d42f665bc848b7b2a98aa9cad7aef2` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:30.618087 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c1064a-218a-4628-bbea-52eb843e8a72 None admin] RBAC: Authorizing `identity:get_user(user_id=50d42f665bc848b7b2a98aa9cad7aef2)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:30.619559 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c1064a-218a-4628-bbea-52eb843e8a72 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:30.621493 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 145/284] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:49:30 2022] GET /identity/v3/users/50d42f665bc848b7b2a98aa9cad7aef2 => generated 297 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:30.626671 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2031db8-4e69-4e6b-996c-f8a5fad03b4e None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:30.807068 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2031db8-4e69-4e6b-996c-f8a5fad03b4e None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:30.835367 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:30.852191 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8851ec-c308-4c7c-8f0a-702d3ccb4e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:30.853112 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8851ec-c308-4c7c-8f0a-702d3ccb4e8e None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:30.853394 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8851ec-c308-4c7c-8f0a-702d3ccb4e8e None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:30.853714 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8851ec-c308-4c7c-8f0a-702d3ccb4e8e None admin] PATH_INFO: `/v3/projects/e8ca129f01d149b5aab391353f90139b` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:30.860048 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8851ec-c308-4c7c-8f0a-702d3ccb4e8e None admin] RBAC: Authorizing `identity:get_project(project_id=e8ca129f01d149b5aab391353f90139b)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:30.861494 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8851ec-c308-4c7c-8f0a-702d3ccb4e8e None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:30.864161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 140/285] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:49:30 2022] GET /identity/v3/projects/e8ca129f01d149b5aab391353f90139b => generated 302 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:30.869586 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c1064a-218a-4628-bbea-52eb843e8a72 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:30.989363 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:31.002259 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c1064a-218a-4628-bbea-52eb843e8a72 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:31.048693 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1934cb8f-2c6d-45c1-bbff-5fd969483af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:31.049604 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934cb8f-2c6d-45c1-bbff-5fd969483af6 None admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:31.049883 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934cb8f-2c6d-45c1-bbff-5fd969483af6 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:31.050155 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934cb8f-2c6d-45c1-bbff-5fd969483af6 None admin] PATH_INFO: `/v3/projects/e8ca129f01d149b5aab391353f90139b/users/50d42f665bc848b7b2a98aa9cad7aef2/roles/30b0c2ba85ff481eb256122140e4163a` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:31.052301 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1934cb8f-2c6d-45c1-bbff-5fd969483af6 None admin] RBAC: Authorizing `identity:create_grant(project_id=e8ca129f01d149b5aab391353f90139b, user_id=50d42f665bc848b7b2a98aa9cad7aef2, role_id=30b0c2ba85ff481eb256122140e4163a)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:31.053773 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1934cb8f-2c6d-45c1-bbff-5fd969483af6 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:31.075006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 146/286] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jun 7 21:49:30 2022] PUT /identity/v3/projects/e8ca129f01d149b5aab391353f90139b/users/50d42f665bc848b7b2a98aa9cad7aef2/roles/30b0c2ba85ff481eb256122140e4163a => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 21:49:31.440243 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc6b70-abda-467f-86c7-2c8f42634378 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:31.440746 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc6b70-abda-467f-86c7-2c8f42634378 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:31.440821 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc6b70-abda-467f-86c7-2c8f42634378 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:31.442850 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 141/287] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:31.448488 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4eef0-e230-46e3-9d26-3637a3f35301 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:31.448779 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4eef0-e230-46e3-9d26-3637a3f35301 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:31.449059 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4eef0-e230-46e3-9d26-3637a3f35301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:31.470251 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-2ab4eef0-e230-46e3-9d26-3637a3f35301 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:31.471484 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-2ab4eef0-e230-46e3-9d26-3637a3f35301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:31.553091 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 147/288] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:49:31.586934 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fc6b70-abda-467f-86c7-2c8f42634378 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:31.721830 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fc6b70-abda-467f-86c7-2c8f42634378 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:31.766193 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037c1cbe-db57-4ba0-88b4-accfdd20368b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:31.766193 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c1cbe-db57-4ba0-88b4-accfdd20368b None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:31.766193 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c1cbe-db57-4ba0-88b4-accfdd20368b None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:31.766193 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c1cbe-db57-4ba0-88b4-accfdd20368b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:31.783971 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037c1cbe-db57-4ba0-88b4-accfdd20368b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:31.796787 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-037c1cbe-db57-4ba0-88b4-accfdd20368b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 21:49:31.796787 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:31.796787 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:31.796787 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:31.796787 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:31.796787 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:31.796787 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:31.796787 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:31.796787 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:31.796787 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:31.796787 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:31.796787 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:31.796787 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:31.796787 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:31.796787 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:31.796787 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:31.797684 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:31.797684 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:31.797684 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:31.797684 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:31.797684 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:31.797684 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:31.797684 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:31.797684 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:31.797684 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:31.797684 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:31.797684 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:31.797684 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:31.797684 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:31.797684 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:31.797684 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:31.797684 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:31.798896 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:31.798896 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:31.798896 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:31.798896 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:31.798896 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:31.798896 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:31.798896 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:31.798896 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:31.798896 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:31.798896 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:31.798896 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 21:49:31.798896 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:49:31.798896 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 142/289] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:49:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:49:31.804314 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab4eef0-e230-46e3-9d26-3637a3f35301 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:31.893759 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:31.913270 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab4eef0-e230-46e3-9d26-3637a3f35301 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:31.957060 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccdd57f-e645-4d16-b9c6-88b90fd61699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:31.957741 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccdd57f-e645-4d16-b9c6-88b90fd61699 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:31.958023 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccdd57f-e645-4d16-b9c6-88b90fd61699 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:31.958276 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccdd57f-e645-4d16-b9c6-88b90fd61699 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:31.959052 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccdd57f-e645-4d16-b9c6-88b90fd61699 None admin] RBAC: Adding query filter params (name=member) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:31.959377 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccdd57f-e645-4d16-b9c6-88b90fd61699 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:31.960804 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccdd57f-e645-4d16-b9c6-88b90fd61699 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:31.970661 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 148/290] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jun 7 21:49:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:31.976620 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037c1cbe-db57-4ba0-88b4-accfdd20368b None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:32.029318 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:32.099835 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037c1cbe-db57-4ba0-88b4-accfdd20368b None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:32.156987 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd24dde6-bb01-4e25-a163-22e05816ae28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:32.157878 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24dde6-bb01-4e25-a163-22e05816ae28 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:32.158164 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24dde6-bb01-4e25-a163-22e05816ae28 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:32.158440 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24dde6-bb01-4e25-a163-22e05816ae28 None admin] PATH_INFO: `/v3/users/3e88180080594ae8b15efcea5ddcfd09` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:32.159706 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd24dde6-bb01-4e25-a163-22e05816ae28 None admin] RBAC: Authorizing `identity:get_user(user_id=3e88180080594ae8b15efcea5ddcfd09)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:32.161706 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd24dde6-bb01-4e25-a163-22e05816ae28 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:32.163586 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 143/291] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:49:31 2022] GET /identity/v3/users/3e88180080594ae8b15efcea5ddcfd09 => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:32.168885 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccdd57f-e645-4d16-b9c6-88b90fd61699 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:32.309542 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccdd57f-e645-4d16-b9c6-88b90fd61699 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:32.347810 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86012d68-b8dd-42b9-84e2-2b0381bc8cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:32.348459 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86012d68-b8dd-42b9-84e2-2b0381bc8cc1 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:32.348740 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86012d68-b8dd-42b9-84e2-2b0381bc8cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:32.349012 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86012d68-b8dd-42b9-84e2-2b0381bc8cc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:32.365208 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:32.421636 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86012d68-b8dd-42b9-84e2-2b0381bc8cc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:32.423145 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86012d68-b8dd-42b9-84e2-2b0381bc8cc1 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:32.524247 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 149/292] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:49:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 21:49:32.529497 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd24dde6-bb01-4e25-a163-22e05816ae28 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:32.631441 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd24dde6-bb01-4e25-a163-22e05816ae28 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:32.648252 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:32.666146 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b2f288-e8a9-4259-a0dd-bce11b9d4028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:32.667013 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2f288-e8a9-4259-a0dd-bce11b9d4028 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:32.667367 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2f288-e8a9-4259-a0dd-bce11b9d4028 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:32.667650 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2f288-e8a9-4259-a0dd-bce11b9d4028 None admin] PATH_INFO: `/v3/projects/e8ca129f01d149b5aab391353f90139b` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:32.668811 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b2f288-e8a9-4259-a0dd-bce11b9d4028 None admin] RBAC: Authorizing `identity:get_project(project_id=e8ca129f01d149b5aab391353f90139b)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:32.670268 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b2f288-e8a9-4259-a0dd-bce11b9d4028 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:32.672287 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 144/293] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:49:32 2022] GET /identity/v3/projects/e8ca129f01d149b5aab391353f90139b => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:32.678701 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86012d68-b8dd-42b9-84e2-2b0381bc8cc1 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:32.692531 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa162869-6ba0-4f42-b0b0-a8771c9bf883 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:32.693173 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa162869-6ba0-4f42-b0b0-a8771c9bf883 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:32.693685 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa162869-6ba0-4f42-b0b0-a8771c9bf883 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:32.695561 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 145/294] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:32.701224 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0719d7da-7385-49a0-9b39-908f64e34c77 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:32.701747 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0719d7da-7385-49a0-9b39-908f64e34c77 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:32.702227 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0719d7da-7385-49a0-9b39-908f64e34c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:32.728868 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:32.740388 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-0719d7da-7385-49a0-9b39-908f64e34c77 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:32.741603 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-0719d7da-7385-49a0-9b39-908f64e34c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:32.790057 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86012d68-b8dd-42b9-84e2-2b0381bc8cc1 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:32.826961 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 146/295] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 21:49:32.827174 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec1c7ce-0611-4e0c-850f-5696628487c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:32.827758 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1c7ce-0611-4e0c-850f-5696628487c7 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:32.828088 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1c7ce-0611-4e0c-850f-5696628487c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:32.828324 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e21531-5cf5-41ab-9c16-b91a241a39a5 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:32.828425 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1c7ce-0611-4e0c-850f-5696628487c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:32.828613 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e21531-5cf5-41ab-9c16-b91a241a39a5 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:32.829023 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e21531-5cf5-41ab-9c16-b91a241a39a5 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:32.829203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec1c7ce-0611-4e0c-850f-5696628487c7 None admin] RBAC: Adding query filter params (role.id=c5431c9deadf47e496264ba5b78eeaf6, scope.project.id=e8ca129f01d149b5aab391353f90139b, user.id=3e88180080594ae8b15efcea5ddcfd09) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:32.829510 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec1c7ce-0611-4e0c-850f-5696628487c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:32.830304 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 147/296] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:32.831212 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec1c7ce-0611-4e0c-850f-5696628487c7 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:32.836503 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec59c7-56cd-4bf6-82cc-5d2ae538f13f None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:32.836789 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec59c7-56cd-4bf6-82cc-5d2ae538f13f None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:32.837068 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec59c7-56cd-4bf6-82cc-5d2ae538f13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:32.838286 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 150/297] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:49:32 2022] GET /identity/v3/role_assignments?user.id=3e88180080594ae8b15efcea5ddcfd09&scope.project.id=e8ca129f01d149b5aab391353f90139b&role.id=c5431c9deadf47e496264ba5b78eeaf6 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:32.854263 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec1c7ce-0611-4e0c-850f-5696628487c7 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:32.856655 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-6bec59c7-56cd-4bf6-82cc-5d2ae538f13f None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:32.858047 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-6bec59c7-56cd-4bf6-82cc-5d2ae538f13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:32.935267 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 148/298] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 21:49:32.944898 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8beb60-871f-493b-b055-40d7ce12b8ac None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:32.945459 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8beb60-871f-493b-b055-40d7ce12b8ac None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:32.945889 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8beb60-871f-493b-b055-40d7ce12b8ac None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:32.947480 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 149/299] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:32 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:32.952961 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75d9e1-619a-4dda-99be-dee4294e5e61 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:32.953450 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75d9e1-619a-4dda-99be-dee4294e5e61 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:32.953895 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75d9e1-619a-4dda-99be-dee4294e5e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:32.970716 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec1c7ce-0611-4e0c-850f-5696628487c7 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:32.981955 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-aa75d9e1-619a-4dda-99be-dee4294e5e61 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:32.982980 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:32.983185 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-aa75d9e1-619a-4dda-99be-dee4294e5e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:33.013992 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc09b3-887b-4a61-b719-62342f36a7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:33.015093 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc09b3-887b-4a61-b719-62342f36a7b2 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:33.015557 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc09b3-887b-4a61-b719-62342f36a7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:33.016022 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc09b3-887b-4a61-b719-62342f36a7b2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:33.028966 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cc09b3-887b-4a61-b719-62342f36a7b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:33.035701 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:33.037731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-81cc09b3-887b-4a61-b719-62342f36a7b2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 21:49:33.037731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:33.037731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:33.037731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:33.037731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:33.037731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:33.037731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:33.037731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:33.037731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:33.037731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:33.037731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:33.037731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:33.037731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:33.037731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:33.037731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:33.037731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:33.038878 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:33.038878 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:33.038878 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:33.038878 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:33.038878 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:33.038878 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:33.038878 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:33.038878 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:33.038878 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:33.038878 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:33.038878 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:33.038878 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:33.038878 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:33.038878 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:33.038878 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:33.038878 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:33.039831 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:33.039831 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:33.039831 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:33.039831 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:33.039831 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:33.039831 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:33.039831 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:33.039831 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:33.039831 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:33.039831 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:33.039831 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 21:49:33.039831 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:49:33.042137 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 151/300] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:49:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:49:33.043113 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc09b3-887b-4a61-b719-62342f36a7b2 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:33.066850 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 150/301] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:49:33.067953 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa75d9e1-619a-4dda-99be-dee4294e5e61 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:33.154016 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc09b3-887b-4a61-b719-62342f36a7b2 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:33.181820 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa75d9e1-619a-4dda-99be-dee4294e5e61 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:33.190491 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe1fea8-705a-4f1b-abce-d36b60d4ef2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:33.192037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1fea8-705a-4f1b-abce-d36b60d4ef2c None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:33.192428 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1fea8-705a-4f1b-abce-d36b60d4ef2c None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:33.192823 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1fea8-705a-4f1b-abce-d36b60d4ef2c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:33.204659 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe1fea8-705a-4f1b-abce-d36b60d4ef2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:33.212310 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-fbe1fea8-705a-4f1b-abce-d36b60d4ef2c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 21:49:33.212310 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:33.212310 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:33.212310 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:33.212310 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:33.212310 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:33.212310 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:33.212310 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:33.212310 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:33.212310 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:33.212310 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:33.212310 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:33.212310 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:33.212310 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:33.212310 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:33.212310 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:33.213210 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:33.213210 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:33.213210 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:33.213210 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:33.213210 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:33.213210 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:33.213210 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:33.213210 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:33.213210 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:33.213210 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:33.213210 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:33.213210 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:33.213210 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:33.213210 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:33.213210 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:33.213210 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:33.214005 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:33.214005 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:33.214005 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:33.214005 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:33.214005 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:33.214005 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:33.214005 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:33.214005 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:33.214005 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:33.214005 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:33.214005 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 21:49:33.214005 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:49:33.214005 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 152/302] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:49:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:49:33.214661 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe1fea8-705a-4f1b-abce-d36b60d4ef2c None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:33.225198 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac00d63-ec5d-4c7c-aac8-0f3d918b1eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:33.225887 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac00d63-ec5d-4c7c-aac8-0f3d918b1eb7 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:33.226172 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac00d63-ec5d-4c7c-aac8-0f3d918b1eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:33.226451 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac00d63-ec5d-4c7c-aac8-0f3d918b1eb7 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:33.227212 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac00d63-ec5d-4c7c-aac8-0f3d918b1eb7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:33.227561 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac00d63-ec5d-4c7c-aac8-0f3d918b1eb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:33.228946 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac00d63-ec5d-4c7c-aac8-0f3d918b1eb7 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:33.237762 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 151/303] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jun 7 21:49:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:33.239213 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac00d63-ec5d-4c7c-aac8-0f3d918b1eb7 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:33.250987 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:33.276307 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:33.331460 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe1fea8-705a-4f1b-abce-d36b60d4ef2c None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:33.344485 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac00d63-ec5d-4c7c-aac8-0f3d918b1eb7 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:33.367917 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9633142-33d7-4374-b9f8-6f47bdff636d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:33.368703 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9633142-33d7-4374-b9f8-6f47bdff636d None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:33.368991 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9633142-33d7-4374-b9f8-6f47bdff636d None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:33.369269 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9633142-33d7-4374-b9f8-6f47bdff636d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:33.370668 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9633142-33d7-4374-b9f8-6f47bdff636d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:33.372151 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9633142-33d7-4374-b9f8-6f47bdff636d None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:33.374004 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 153/304] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:49:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:33.379298 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9633142-33d7-4374-b9f8-6f47bdff636d None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:33.383361 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76debd21-bb43-4483-9133-f9c4e43c5b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:33.384062 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76debd21-bb43-4483-9133-f9c4e43c5b6c None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:33.384314 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76debd21-bb43-4483-9133-f9c4e43c5b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:33.384591 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76debd21-bb43-4483-9133-f9c4e43c5b6c None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:33.385347 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76debd21-bb43-4483-9133-f9c4e43c5b6c None admin] RBAC: Adding query filter params (name=member) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:33.385652 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76debd21-bb43-4483-9133-f9c4e43c5b6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:33.387039 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76debd21-bb43-4483-9133-f9c4e43c5b6c None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:33.396355 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 152/305] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jun 7 21:49:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:33.401681 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76debd21-bb43-4483-9133-f9c4e43c5b6c None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:33.479890 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:33.487650 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9633142-33d7-4374-b9f8-6f47bdff636d None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:33.503482 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:33.508035 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76debd21-bb43-4483-9133-f9c4e43c5b6c None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:33.525026 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f3e854-ab16-4d90-ad7a-6afa449349f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:33.525902 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f3e854-ab16-4d90-ad7a-6afa449349f9 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:33.526190 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f3e854-ab16-4d90-ad7a-6afa449349f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:33.526467 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f3e854-ab16-4d90-ad7a-6afa449349f9 None admin] PATH_INFO: `/v3/users/1f699db471e64cc6ad48d6410e36184b` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:33.541983 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f3e854-ab16-4d90-ad7a-6afa449349f9 None admin] RBAC: Authorizing `identity:get_user(user_id=1f699db471e64cc6ad48d6410e36184b)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:33.543521 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f3e854-ab16-4d90-ad7a-6afa449349f9 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:33.544918 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d8a426-e089-45f7-ae4b-605515e9a769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:33.545710 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8a426-e089-45f7-ae4b-605515e9a769 None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:33.545999 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8a426-e089-45f7-ae4b-605515e9a769 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:33.546274 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8a426-e089-45f7-ae4b-605515e9a769 None admin] PATH_INFO: `/v3/groups` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:33.546983 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d8a426-e089-45f7-ae4b-605515e9a769 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:33.548509 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d8a426-e089-45f7-ae4b-605515e9a769 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:33.550500 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 154/306] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:49:33 2022] GET /identity/v3/users/1f699db471e64cc6ad48d6410e36184b => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:33.551943 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f3e854-ab16-4d90-ad7a-6afa449349f9 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:33.566487 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 153/307] 10.222.0.13 () {60 vars in 1134 bytes} [Tue Jun 7 21:49:33 2022] POST /identity/v3/groups => generated 229 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 21:49:33.572440 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d8a426-e089-45f7-ae4b-605515e9a769 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:33.681266 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f3e854-ab16-4d90-ad7a-6afa449349f9 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:33.724496 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b857233-1846-4675-aff3-b50c7a8efbb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:33.725382 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b857233-1846-4675-aff3-b50c7a8efbb6 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:33.725660 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b857233-1846-4675-aff3-b50c7a8efbb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:33.726174 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b857233-1846-4675-aff3-b50c7a8efbb6 None admin] PATH_INFO: `/v3/users/316a5a2476284a778d902dca87bae5c6` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:33.728077 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b857233-1846-4675-aff3-b50c7a8efbb6 None admin] RBAC: Authorizing `identity:get_user(user_id=316a5a2476284a778d902dca87bae5c6)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:33.729431 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b857233-1846-4675-aff3-b50c7a8efbb6 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:33.731444 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 155/308] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:49:33 2022] GET /identity/v3/users/316a5a2476284a778d902dca87bae5c6 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:33.736609 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d8a426-e089-45f7-ae4b-605515e9a769 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:33.737207 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b857233-1846-4675-aff3-b50c7a8efbb6 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:33.774201 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:33.789469 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac3c0d3-9a43-4fd4-92cf-a508f0983643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:33.790141 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3c0d3-9a43-4fd4-92cf-a508f0983643 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:33.790426 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3c0d3-9a43-4fd4-92cf-a508f0983643 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:33.790731 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3c0d3-9a43-4fd4-92cf-a508f0983643 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:33.792823 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac3c0d3-9a43-4fd4-92cf-a508f0983643 None admin] RBAC: Adding query filter params (role.id=30b0c2ba85ff481eb256122140e4163a, user.id=1f699db471e64cc6ad48d6410e36184b, scope.system=all) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:33.793141 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac3c0d3-9a43-4fd4-92cf-a508f0983643 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:33.794580 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac3c0d3-9a43-4fd4-92cf-a508f0983643 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:33.804981 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 154/309] 10.222.0.13 () {56 vars in 1308 bytes} [Tue Jun 7 21:49:33 2022] GET /identity/v3/role_assignments?user.id=1f699db471e64cc6ad48d6410e36184b&scope.system=all&role.id=30b0c2ba85ff481eb256122140e4163a => generated 226 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:33.879690 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b857233-1846-4675-aff3-b50c7a8efbb6 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:33.897379 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e009fc2-95d8-41eb-be2e-082e4e046661 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:33.897669 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e009fc2-95d8-41eb-be2e-082e4e046661 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:33.897953 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e009fc2-95d8-41eb-be2e-082e4e046661 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:33.899500 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 155/310] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 21:49:33.904953 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f126d09-bdd0-441e-977a-9f88afbf9cdc None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:33.905333 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f126d09-bdd0-441e-977a-9f88afbf9cdc None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:33.905677 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f126d09-bdd0-441e-977a-9f88afbf9cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:33.932603 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-9f126d09-bdd0-441e-977a-9f88afbf9cdc None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:33.934303 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-9f126d09-bdd0-441e-977a-9f88afbf9cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:33.934943 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5bcb18-15e8-4e24-ab1e-532468df168a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:33.935934 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5bcb18-15e8-4e24-ab1e-532468df168a None admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:33.936281 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5bcb18-15e8-4e24-ab1e-532468df168a None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:33.936625 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5bcb18-15e8-4e24-ab1e-532468df168a None admin] PATH_INFO: `/v3/system/users/316a5a2476284a778d902dca87bae5c6/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:33.938253 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5bcb18-15e8-4e24-ab1e-532468df168a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=316a5a2476284a778d902dca87bae5c6, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:33.939781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5bcb18-15e8-4e24-ab1e-532468df168a None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:33.950814 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 156/311] 10.222.0.13 () {58 vars in 1334 bytes} [Tue Jun 7 21:49:33 2022] PUT /identity/v3/system/users/316a5a2476284a778d902dca87bae5c6/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 21:49:33.958609 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:34.031088 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 156/312] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:49:34.055654 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5bcb18-15e8-4e24-ab1e-532468df168a None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:34.155278 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:34.171966 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5bcb18-15e8-4e24-ab1e-532468df168a None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:34.208705 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84551c55-ddf0-4d34-8f8e-5c5b0a5135f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:34.209512 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84551c55-ddf0-4d34-8f8e-5c5b0a5135f9 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:34.209798 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84551c55-ddf0-4d34-8f8e-5c5b0a5135f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:34.210072 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84551c55-ddf0-4d34-8f8e-5c5b0a5135f9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:34.220935 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84551c55-ddf0-4d34-8f8e-5c5b0a5135f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:34.228976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-84551c55-ddf0-4d34-8f8e-5c5b0a5135f9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 21:49:34.228976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:34.228976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:34.228976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:34.228976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:34.228976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:34.228976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:34.228976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:34.228976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:34.228976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:34.228976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:34.228976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:34.228976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:34.228976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:34.228976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:34.228976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:34.229867 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:34.229867 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:34.229867 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:34.229867 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:34.229867 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:34.229867 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:34.229867 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:34.229867 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:34.229867 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:34.229867 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:34.229867 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:34.229867 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:34.229867 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:34.229867 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:34.229867 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:34.229867 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:34.230642 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:34.230642 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:34.230642 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:34.230642 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:34.230642 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:34.230642 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:34.230642 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:34.230642 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:34.230642 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:34.230642 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:34.230642 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 21:49:34.230642 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:49:34.230642 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 157/313] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:49:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:49:34.235968 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f126d09-bdd0-441e-977a-9f88afbf9cdc None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:34.348314 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f126d09-bdd0-441e-977a-9f88afbf9cdc None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:34.386167 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9688ed6e-9377-462f-95b9-2136d1918917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:34.386842 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9688ed6e-9377-462f-95b9-2136d1918917 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:34.387126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9688ed6e-9377-462f-95b9-2136d1918917 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:34.387507 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9688ed6e-9377-462f-95b9-2136d1918917 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:34.388267 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9688ed6e-9377-462f-95b9-2136d1918917 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:34.388572 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9688ed6e-9377-462f-95b9-2136d1918917 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:34.389940 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9688ed6e-9377-462f-95b9-2136d1918917 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:34.398602 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 157/314] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jun 7 21:49:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:34.404003 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84551c55-ddf0-4d34-8f8e-5c5b0a5135f9 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:34.516231 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84551c55-ddf0-4d34-8f8e-5c5b0a5135f9 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:34.556266 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8a733-6e5b-46d2-b73b-2e3f6b9b587e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:34.557145 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8a733-6e5b-46d2-b73b-2e3f6b9b587e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:34.557474 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8a733-6e5b-46d2-b73b-2e3f6b9b587e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:34.557754 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8a733-6e5b-46d2-b73b-2e3f6b9b587e None admin] PATH_INFO: `/v3/users/50d42f665bc848b7b2a98aa9cad7aef2` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:34.558950 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f8a733-6e5b-46d2-b73b-2e3f6b9b587e None admin] RBAC: Authorizing `identity:get_user(user_id=50d42f665bc848b7b2a98aa9cad7aef2)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:34.560438 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f8a733-6e5b-46d2-b73b-2e3f6b9b587e None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:34.562362 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 158/315] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:49:34 2022] GET /identity/v3/users/50d42f665bc848b7b2a98aa9cad7aef2 => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:34.568381 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9688ed6e-9377-462f-95b9-2136d1918917 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:34.591715 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:34.673421 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9688ed6e-9377-462f-95b9-2136d1918917 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:34.708483 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2fcf21-bbf0-4315-9a68-2ae6e65c1913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:34.709140 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2fcf21-bbf0-4315-9a68-2ae6e65c1913 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:34.709424 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2fcf21-bbf0-4315-9a68-2ae6e65c1913 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:34.709700 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2fcf21-bbf0-4315-9a68-2ae6e65c1913 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:34.780620 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2fcf21-bbf0-4315-9a68-2ae6e65c1913 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:34.782100 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2fcf21-bbf0-4315-9a68-2ae6e65c1913 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:34.819735 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:34.887843 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 158/316] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:49:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 21:49:34.893486 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8a733-6e5b-46d2-b73b-2e3f6b9b587e None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:34.911018 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:34.998856 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8a733-6e5b-46d2-b73b-2e3f6b9b587e None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:35.035004 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce8f3ef-20b8-4052-ae5f-2ebe38123614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:35.035920 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce8f3ef-20b8-4052-ae5f-2ebe38123614 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:35.036204 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce8f3ef-20b8-4052-ae5f-2ebe38123614 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:35.036483 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce8f3ef-20b8-4052-ae5f-2ebe38123614 None admin] PATH_INFO: `/v3/projects/e8ca129f01d149b5aab391353f90139b` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:35.037722 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce8f3ef-20b8-4052-ae5f-2ebe38123614 None admin] RBAC: Authorizing `identity:get_project(project_id=e8ca129f01d149b5aab391353f90139b)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:35.039282 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce8f3ef-20b8-4052-ae5f-2ebe38123614 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:35.041298 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 159/317] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:49:34 2022] GET /identity/v3/projects/e8ca129f01d149b5aab391353f90139b => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:35.047032 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2fcf21-bbf0-4315-9a68-2ae6e65c1913 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:35.161854 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2fcf21-bbf0-4315-9a68-2ae6e65c1913 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:35.179117 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:35.200461 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4824d2-350a-4e01-85cc-2dc3fc1993d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:35.201119 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4824d2-350a-4e01-85cc-2dc3fc1993d4 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:35.201462 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4824d2-350a-4e01-85cc-2dc3fc1993d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:35.201742 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4824d2-350a-4e01-85cc-2dc3fc1993d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:35.202623 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4824d2-350a-4e01-85cc-2dc3fc1993d4 None admin] RBAC: Adding query filter params (role.id=30b0c2ba85ff481eb256122140e4163a, scope.project.id=e8ca129f01d149b5aab391353f90139b, user.id=50d42f665bc848b7b2a98aa9cad7aef2) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:35.202937 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4824d2-350a-4e01-85cc-2dc3fc1993d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:35.204473 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4824d2-350a-4e01-85cc-2dc3fc1993d4 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:35.211534 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 159/318] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:49:35 2022] GET /identity/v3/role_assignments?user.id=50d42f665bc848b7b2a98aa9cad7aef2&scope.project.id=e8ca129f01d149b5aab391353f90139b&role.id=30b0c2ba85ff481eb256122140e4163a => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:35.970334 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b3b4f-3f59-4a9d-ad47-672dcedad98c None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:35.971292 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b3b4f-3f59-4a9d-ad47-672dcedad98c None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:35.972071 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b3b4f-3f59-4a9d-ad47-672dcedad98c None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:35.974396 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 160/319] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:35.979522 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f75aefd-f6ca-4d44-9602-08cd65630427 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:35.980037 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f75aefd-f6ca-4d44-9602-08cd65630427 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:35.980116 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f75aefd-f6ca-4d44-9602-08cd65630427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:35.998114 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-4f75aefd-f6ca-4d44-9602-08cd65630427 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:35.999373 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-4f75aefd-f6ca-4d44-9602-08cd65630427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:36.071660 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 160/320] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:49:36.094234 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b3b4f-3f59-4a9d-ad47-672dcedad98c None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:36.178734 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471de02-98c0-487c-be6c-dd1622dfc140 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:36.179058 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471de02-98c0-487c-be6c-dd1622dfc140 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:36.179291 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471de02-98c0-487c-be6c-dd1622dfc140 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:36.180851 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 161/321] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 21:49:36.185563 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27468fbf-a451-4fca-94d5-9d4ca63c180e None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:36.185846 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27468fbf-a451-4fca-94d5-9d4ca63c180e None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:36.186121 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27468fbf-a451-4fca-94d5-9d4ca63c180e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:36.186901 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:36.204735 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-27468fbf-a451-4fca-94d5-9d4ca63c180e None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:36.205913 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-27468fbf-a451-4fca-94d5-9d4ca63c180e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:36.212972 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b3b4f-3f59-4a9d-ad47-672dcedad98c None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:36.255642 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8974630c-1a38-4b01-b42c-407cfe5f51de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:36.256471 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8974630c-1a38-4b01-b42c-407cfe5f51de None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:36.256973 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8974630c-1a38-4b01-b42c-407cfe5f51de None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:36.257252 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8974630c-1a38-4b01-b42c-407cfe5f51de None admin] PATH_INFO: `/v3/domains/default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:36.259319 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8974630c-1a38-4b01-b42c-407cfe5f51de None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:36.261408 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8974630c-1a38-4b01-b42c-407cfe5f51de None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:36.267499 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 161/322] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:49:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:36.270086 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8974630c-1a38-4b01-b42c-407cfe5f51de None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:36.283294 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 162/323] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 21:49:36.307231 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27468fbf-a451-4fca-94d5-9d4ca63c180e None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:36.350251 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:36.381389 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8974630c-1a38-4b01-b42c-407cfe5f51de None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:36.409521 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27468fbf-a451-4fca-94d5-9d4ca63c180e None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:36.413579 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d9a8b2-fff7-44a9-ae5f-d262519932d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:36.414240 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9a8b2-fff7-44a9-ae5f-d262519932d3 None admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:36.414521 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9a8b2-fff7-44a9-ae5f-d262519932d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:36.414796 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9a8b2-fff7-44a9-ae5f-d262519932d3 None admin] PATH_INFO: `/v3/groups` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:36.415509 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d9a8b2-fff7-44a9-ae5f-d262519932d3 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:36.416932 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d9a8b2-fff7-44a9-ae5f-d262519932d3 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:36.439472 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 162/324] 10.222.0.13 () {60 vars in 1134 bytes} [Tue Jun 7 21:49:36 2022] POST /identity/v3/groups => generated 226 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 21:49:36.439472 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92094e-a1b6-4043-b33d-9496a7221f6d None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:36.439472 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92094e-a1b6-4043-b33d-9496a7221f6d None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:36.439472 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92094e-a1b6-4043-b33d-9496a7221f6d None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:36.439472 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 163/325] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:36.451362 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3af497-e6df-4a5d-a77d-8b16c963c5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:36.451362 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3af497-e6df-4a5d-a77d-8b16c963c5d7 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:36.451362 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3af497-e6df-4a5d-a77d-8b16c963c5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:36.451698 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a3b3b-b5fb-4ddf-9958-5e20e1d53cf6 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:36.451771 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3af497-e6df-4a5d-a77d-8b16c963c5d7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:36.451872 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a3b3b-b5fb-4ddf-9958-5e20e1d53cf6 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:36.451872 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a3b3b-b5fb-4ddf-9958-5e20e1d53cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:36.460260 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3af497-e6df-4a5d-a77d-8b16c963c5d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:36.468432 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-c81a3b3b-b5fb-4ddf-9958-5e20e1d53cf6 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:36.468639 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-6f3af497-e6df-4a5d-a77d-8b16c963c5d7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 21:49:36.468639 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:36.468639 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:36.468639 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:36.468639 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:36.468639 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:36.468639 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:36.468639 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:36.468639 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:36.468639 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:36.468639 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:36.468639 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:36.468639 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:36.468639 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:36.468639 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:36.468639 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:36.469495 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:36.469495 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:36.469495 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:36.469495 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:36.469495 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:36.469495 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:36.469495 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:36.469495 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:36.469495 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:36.469495 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:36.469495 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:36.469495 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:36.469495 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:36.469495 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:36.469495 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:36.469495 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:36.470302 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:36.470302 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:36.470302 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:36.470302 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:36.470302 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:36.470302 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:36.470302 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:36.470302 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:36.470302 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:36.470302 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:36.470302 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 21:49:36.470302 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:49:36.470916 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c81a3b3b-b5fb-4ddf-9958-5e20e1d53cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:36.470984 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 163/326] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:49:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:49:36.473496 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:36.477928 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3af497-e6df-4a5d-a77d-8b16c963c5d7 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:36.549870 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 164/327] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 21:49:36.574830 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a3b3b-b5fb-4ddf-9958-5e20e1d53cf6 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:36.594804 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:36.599610 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3af497-e6df-4a5d-a77d-8b16c963c5d7 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:36.641790 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b0b71-a3ce-42ed-9508-ebbf10312ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:36.642391 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b0b71-a3ce-42ed-9508-ebbf10312ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:36.642677 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b0b71-a3ce-42ed-9508-ebbf10312ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:36.642956 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b0b71-a3ce-42ed-9508-ebbf10312ab6 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:36.643756 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7b0b71-a3ce-42ed-9508-ebbf10312ab6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:36.644105 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7b0b71-a3ce-42ed-9508-ebbf10312ab6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:36.646021 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7b0b71-a3ce-42ed-9508-ebbf10312ab6 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:36.655832 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 164/328] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jun 7 21:49:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:36.661578 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b0b71-a3ce-42ed-9508-ebbf10312ab6 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:36.693969 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a3b3b-b5fb-4ddf-9958-5e20e1d53cf6 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:36.739937 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce550394-23d6-4c75-ad17-01003002ad8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:36.740984 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce550394-23d6-4c75-ad17-01003002ad8b None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:36.741437 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce550394-23d6-4c75-ad17-01003002ad8b None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:36.741853 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce550394-23d6-4c75-ad17-01003002ad8b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:36.756999 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:36.767467 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce550394-23d6-4c75-ad17-01003002ad8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:36.771277 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-ce550394-23d6-4c75-ad17-01003002ad8b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 21:49:36.771277 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:36.771277 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:36.771277 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:36.771277 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:36.771277 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:36.771277 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:36.771277 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:36.771277 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:36.771277 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:36.771277 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:36.771277 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:36.771277 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:36.771277 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:36.771277 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:36.771277 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:36.772223 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:36.772223 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:36.772223 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:36.772223 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:36.772223 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:36.772223 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:36.772223 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:36.772223 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:36.772223 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:36.772223 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:36.772223 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:36.772223 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:36.772223 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:36.772223 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:36.772223 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:36.772223 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:36.773043 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:36.773043 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:36.773043 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:36.773043 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:36.773043 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:36.773043 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:36.773043 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:36.773043 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:36.773043 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:36.773043 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:36.773043 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 21:49:36.773043 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:49:36.773043 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 165/329] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:49:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:49:36.780004 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce550394-23d6-4c75-ad17-01003002ad8b None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:36.787132 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b0b71-a3ce-42ed-9508-ebbf10312ab6 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:36.829481 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed6192-118d-4483-8e35-f00825fec18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:36.830825 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed6192-118d-4483-8e35-f00825fec18e None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:36.830825 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed6192-118d-4483-8e35-f00825fec18e None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:36.830994 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed6192-118d-4483-8e35-f00825fec18e None admin] PATH_INFO: `/v3/users/1f699db471e64cc6ad48d6410e36184b` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:36.832316 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed6192-118d-4483-8e35-f00825fec18e None admin] RBAC: Authorizing `identity:get_user(user_id=1f699db471e64cc6ad48d6410e36184b)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:36.833778 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed6192-118d-4483-8e35-f00825fec18e None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:36.835751 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 165/330] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:49:36 2022] GET /identity/v3/users/1f699db471e64cc6ad48d6410e36184b => generated 293 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:49:36.843252 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed6192-118d-4483-8e35-f00825fec18e None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:36.879458 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:36.922963 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce550394-23d6-4c75-ad17-01003002ad8b None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:36.967008 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761b48e4-e2d6-4a1f-b85b-7a47e3a13458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:36.967861 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b48e4-e2d6-4a1f-b85b-7a47e3a13458 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:36.968380 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b48e4-e2d6-4a1f-b85b-7a47e3a13458 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:36.968711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b48e4-e2d6-4a1f-b85b-7a47e3a13458 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:36.969674 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761b48e4-e2d6-4a1f-b85b-7a47e3a13458 None admin] RBAC: Adding query filter params (name=member) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:36.970107 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761b48e4-e2d6-4a1f-b85b-7a47e3a13458 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:36.971870 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761b48e4-e2d6-4a1f-b85b-7a47e3a13458 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:36.977389 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed6192-118d-4483-8e35-f00825fec18e None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:36.981977 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 166/331] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jun 7 21:49:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:36.988920 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761b48e4-e2d6-4a1f-b85b-7a47e3a13458 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:37.017444 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a999b-c067-4153-a7fd-39707654c55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:37.018915 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a999b-c067-4153-a7fd-39707654c55f None admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:37.019996 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a999b-c067-4153-a7fd-39707654c55f None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:37.020421 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a999b-c067-4153-a7fd-39707654c55f None admin] PATH_INFO: `/v3/system/users/1f699db471e64cc6ad48d6410e36184b/roles/30b0c2ba85ff481eb256122140e4163a` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:37.022245 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9a999b-c067-4153-a7fd-39707654c55f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1f699db471e64cc6ad48d6410e36184b, role_id=30b0c2ba85ff481eb256122140e4163a)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:37.024056 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9a999b-c067-4153-a7fd-39707654c55f None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:37.035811 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 166/332] 10.222.0.13 () {58 vars in 1334 bytes} [Tue Jun 7 21:49:36 2022] PUT /identity/v3/system/users/1f699db471e64cc6ad48d6410e36184b/roles/30b0c2ba85ff481eb256122140e4163a => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 21:49:37.064538 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:37.111608 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761b48e4-e2d6-4a1f-b85b-7a47e3a13458 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:37.150873 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff21e5-0215-4163-bbd1-2df857663dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:37.151846 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff21e5-0215-4163-bbd1-2df857663dd4 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:37.152132 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff21e5-0215-4163-bbd1-2df857663dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:37.152409 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff21e5-0215-4163-bbd1-2df857663dd4 None admin] PATH_INFO: `/v3/users/316a5a2476284a778d902dca87bae5c6` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:37.153666 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ff21e5-0215-4163-bbd1-2df857663dd4 None admin] RBAC: Authorizing `identity:get_user(user_id=316a5a2476284a778d902dca87bae5c6)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:37.155130 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ff21e5-0215-4163-bbd1-2df857663dd4 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:37.157271 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 167/333] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:49:36 2022] GET /identity/v3/users/316a5a2476284a778d902dca87bae5c6 => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:37.163994 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a999b-c067-4153-a7fd-39707654c55f None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:37.277246 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:37.289182 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a999b-c067-4153-a7fd-39707654c55f None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:37.330185 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f290d032-83bc-4e64-aa3a-99de21cab808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:37.330886 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290d032-83bc-4e64-aa3a-99de21cab808 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:37.331172 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290d032-83bc-4e64-aa3a-99de21cab808 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:37.332069 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290d032-83bc-4e64-aa3a-99de21cab808 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:37.333226 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f290d032-83bc-4e64-aa3a-99de21cab808 None admin] RBAC: Adding query filter params (role.id=c5431c9deadf47e496264ba5b78eeaf6, user.id=316a5a2476284a778d902dca87bae5c6, scope.system=all) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:37.333543 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f290d032-83bc-4e64-aa3a-99de21cab808 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:37.335068 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f290d032-83bc-4e64-aa3a-99de21cab808 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:37.349655 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 167/334] 10.222.0.13 () {56 vars in 1308 bytes} [Tue Jun 7 21:49:37 2022] GET /identity/v3/role_assignments?user.id=316a5a2476284a778d902dca87bae5c6&scope.system=all&role.id=c5431c9deadf47e496264ba5b78eeaf6 => generated 510 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:39.080988 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e0480-aebb-490b-aa16-7b9058930bf6 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:39.082107 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e0480-aebb-490b-aa16-7b9058930bf6 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:39.082604 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e0480-aebb-490b-aa16-7b9058930bf6 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:39.084490 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 168/335] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:39.090114 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e7abf4-ddf2-4a66-82fd-32b59987bd68 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:39.090565 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e7abf4-ddf2-4a66-82fd-32b59987bd68 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:39.090749 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e7abf4-ddf2-4a66-82fd-32b59987bd68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:39.116532 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-50e7abf4-ddf2-4a66-82fd-32b59987bd68 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:39.117793 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-50e7abf4-ddf2-4a66-82fd-32b59987bd68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:39.218998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 168/336] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:49:39.264915 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e0480-aebb-490b-aa16-7b9058930bf6 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:39.409369 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e0480-aebb-490b-aa16-7b9058930bf6 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:39.440717 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:39.446790 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7070dfd7-20a9-421e-aac3-bcb7776df69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:39.447649 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070dfd7-20a9-421e-aac3-bcb7776df69a None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:39.447932 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070dfd7-20a9-421e-aac3-bcb7776df69a None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:39.448241 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070dfd7-20a9-421e-aac3-bcb7776df69a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:39.461255 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7070dfd7-20a9-421e-aac3-bcb7776df69a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:39.474490 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-7070dfd7-20a9-421e-aac3-bcb7776df69a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 21:49:39.474490 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:39.474490 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:39.474490 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:39.474490 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:39.474490 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:39.474490 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:39.474490 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:39.474490 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:39.474490 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:39.474490 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:39.474490 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:39.474490 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:39.474490 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:39.474490 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:39.474490 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:39.475541 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:39.475541 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:39.475541 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:39.475541 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:39.475541 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:39.475541 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:39.475541 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:39.475541 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:39.475541 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:39.475541 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:39.475541 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:39.475541 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:39.475541 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:39.475541 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:39.475541 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:39.475541 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:39.476379 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:39.476379 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:39.476379 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:39.476379 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:39.476379 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:39.476379 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:39.476379 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:39.476379 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:39.476379 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:39.476379 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:39.476379 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 21:49:39.476379 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:49:39.476379 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 169/337] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:49:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:49:39.481908 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e7abf4-ddf2-4a66-82fd-32b59987bd68 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:39.582505 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:39.607058 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e7abf4-ddf2-4a66-82fd-32b59987bd68 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:39.649823 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aeb049-c36e-4893-83cc-aaac9642928d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:39.650509 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aeb049-c36e-4893-83cc-aaac9642928d None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:39.650793 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aeb049-c36e-4893-83cc-aaac9642928d None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:39.651076 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aeb049-c36e-4893-83cc-aaac9642928d None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:39.651861 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aeb049-c36e-4893-83cc-aaac9642928d None admin] RBAC: Adding query filter params (name=member) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:39.652168 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aeb049-c36e-4893-83cc-aaac9642928d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:39.653552 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aeb049-c36e-4893-83cc-aaac9642928d None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:39.665461 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 169/338] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jun 7 21:49:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:39.670844 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7070dfd7-20a9-421e-aac3-bcb7776df69a None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:39.798494 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7070dfd7-20a9-421e-aac3-bcb7776df69a None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:39.846988 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a61e4ff-fd4e-418e-a7a0-d34cc56feacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:39.848703 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61e4ff-fd4e-418e-a7a0-d34cc56feacc None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:39.848992 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61e4ff-fd4e-418e-a7a0-d34cc56feacc None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:39.849268 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61e4ff-fd4e-418e-a7a0-d34cc56feacc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:39.929592 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:39.942538 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a61e4ff-fd4e-418e-a7a0-d34cc56feacc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:39.944029 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a61e4ff-fd4e-418e-a7a0-d34cc56feacc None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:39.976979 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e785910-6b0e-4409-aba1-9d0d49696249 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:39.977303 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e785910-6b0e-4409-aba1-9d0d49696249 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:39.977554 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e785910-6b0e-4409-aba1-9d0d49696249 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:39.979220 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 170/339] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:39.984356 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32c71e-d159-45cc-8766-8986a022926e None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:39.984651 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32c71e-d159-45cc-8766-8986a022926e None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:39.984925 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32c71e-d159-45cc-8766-8986a022926e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:40.012644 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-cf32c71e-d159-45cc-8766-8986a022926e None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:40.013855 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-cf32c71e-d159-45cc-8766-8986a022926e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:40.102682 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 170/340] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:49:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 21:49:40.109650 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a61e4ff-fd4e-418e-a7a0-d34cc56feacc None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:40.163809 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 171/341] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:49:40.165243 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf17df-6670-4000-8266-9b5850ad1ff1 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:40.165536 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf17df-6670-4000-8266-9b5850ad1ff1 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:40.165818 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf17df-6670-4000-8266-9b5850ad1ff1 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:40.167613 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 172/342] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:40.177083 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc899979-19d4-456e-9aa4-67bb6f08d06d None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:40.177369 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc899979-19d4-456e-9aa4-67bb6f08d06d None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:40.177646 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc899979-19d4-456e-9aa4-67bb6f08d06d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:40.208203 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-bc899979-19d4-456e-9aa4-67bb6f08d06d None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:40.209342 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-bc899979-19d4-456e-9aa4-67bb6f08d06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:40.250766 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:40.262457 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a61e4ff-fd4e-418e-a7a0-d34cc56feacc None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:40.325749 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99be7e1c-cdd9-4166-b5a7-b0981da8b3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:40.326619 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be7e1c-cdd9-4166-b5a7-b0981da8b3f8 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:40.326894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be7e1c-cdd9-4166-b5a7-b0981da8b3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:40.327165 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be7e1c-cdd9-4166-b5a7-b0981da8b3f8 None admin] PATH_INFO: `/v3/projects/b1e275c904b64687b29e7ca3d28fdda7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:40.330620 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99be7e1c-cdd9-4166-b5a7-b0981da8b3f8 None admin] RBAC: Authorizing `identity:get_project(project_id=b1e275c904b64687b29e7ca3d28fdda7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:40.334014 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99be7e1c-cdd9-4166-b5a7-b0981da8b3f8 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:40.336025 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 171/343] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:49:40 2022] GET /identity/v3/projects/b1e275c904b64687b29e7ca3d28fdda7 => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:40.336601 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99be7e1c-cdd9-4166-b5a7-b0981da8b3f8 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:40.377105 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 173/344] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:49:40.378271 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc899979-19d4-456e-9aa4-67bb6f08d06d None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:40.386295 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:40.557143 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc899979-19d4-456e-9aa4-67bb6f08d06d None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:40.567920 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99be7e1c-cdd9-4166-b5a7-b0981da8b3f8 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:40.615563 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b66aba-f7a8-4709-87c1-5a65d98e7547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:40.616382 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b66aba-f7a8-4709-87c1-5a65d98e7547 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:40.616660 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b66aba-f7a8-4709-87c1-5a65d98e7547 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:40.616931 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b66aba-f7a8-4709-87c1-5a65d98e7547 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:40.640064 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ccfb76-a499-47a8-a7a3-11d9a7e02bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:40.640932 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ccfb76-a499-47a8-a7a3-11d9a7e02bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:40.641210 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ccfb76-a499-47a8-a7a3-11d9a7e02bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:40.641479 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ccfb76-a499-47a8-a7a3-11d9a7e02bf2 None admin] PATH_INFO: `/v3/groups/e66391be7a2e4d2dab0bec9fcf9bdab7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:40.642524 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b66aba-f7a8-4709-87c1-5a65d98e7547 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:40.657163 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-89b66aba-f7a8-4709-87c1-5a65d98e7547 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 21:49:40.657163 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:40.657163 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:40.657163 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:40.657163 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:40.657163 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:40.657163 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:40.657163 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:40.657163 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:40.657163 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:40.657163 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:40.657163 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:40.657163 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:40.657163 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:40.657163 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:40.657163 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:40.660244 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:40.660244 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:40.660244 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:40.660244 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:40.660244 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:40.660244 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:40.660244 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:40.660244 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:40.660244 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:40.660244 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:40.660244 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:40.660244 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:40.660244 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:40.660244 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:40.660244 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:40.660244 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:40.662142 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:40.662142 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:40.662142 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:40.662142 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:40.662142 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:40.662142 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:40.662142 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:40.662142 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:40.662142 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:40.662142 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:40.662142 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 21:49:40.662142 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:49:40.662142 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 172/345] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Jun 7 21:49:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:49:40.662142 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b66aba-f7a8-4709-87c1-5a65d98e7547 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:40.663878 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ccfb76-a499-47a8-a7a3-11d9a7e02bf2 None admin] RBAC: Authorizing `identity:get_group(group_id=e66391be7a2e4d2dab0bec9fcf9bdab7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:40.663878 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ccfb76-a499-47a8-a7a3-11d9a7e02bf2 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:40.669107 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 174/346] 10.222.0.13 () {56 vars in 1180 bytes} [Tue Jun 7 21:49:40 2022] GET /identity/v3/groups/e66391be7a2e4d2dab0bec9fcf9bdab7 => generated 226 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:40.670053 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ccfb76-a499-47a8-a7a3-11d9a7e02bf2 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:40.694571 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:40.798148 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b66aba-f7a8-4709-87c1-5a65d98e7547 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:40.838097 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f6b7ce-6b23-4f5e-a7ea-4ba5203a694e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:40.838815 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6b7ce-6b23-4f5e-a7ea-4ba5203a694e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:40.839264 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6b7ce-6b23-4f5e-a7ea-4ba5203a694e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:40.840147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6b7ce-6b23-4f5e-a7ea-4ba5203a694e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:40.852824 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f6b7ce-6b23-4f5e-a7ea-4ba5203a694e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:40.862341 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-18f6b7ce-6b23-4f5e-a7ea-4ba5203a694e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 21:49:40.862341 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:40.862341 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:40.862341 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:40.862341 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:40.862341 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:40.862341 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:40.862341 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:40.862341 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:40.862341 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:40.862341 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:40.862341 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:40.862341 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:40.862341 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:40.862341 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:40.862341 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:40.864721 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:40.864721 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:40.864721 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:40.864721 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:40.864721 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:40.864721 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:40.864721 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:40.864721 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:40.864721 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:40.864721 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:40.864721 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:40.864721 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:40.864721 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:40.864721 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:40.864721 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:40.864721 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:40.865731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:40.865731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:40.865731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:40.865731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:40.865731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:40.865731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:40.865731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:40.865731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:40.865731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:40.865731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:40.865731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 21:49:40.865731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:49:40.868064 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 173/347] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:49:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:49:40.868975 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f6b7ce-6b23-4f5e-a7ea-4ba5203a694e None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:40.899313 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:40.938880 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ccfb76-a499-47a8-a7a3-11d9a7e02bf2 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:40.978647 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73971be-21d1-4e2d-a927-8d542160457f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:40.979638 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73971be-21d1-4e2d-a927-8d542160457f None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:40.980143 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73971be-21d1-4e2d-a927-8d542160457f None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:40.980640 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73971be-21d1-4e2d-a927-8d542160457f None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:40.981606 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73971be-21d1-4e2d-a927-8d542160457f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:40.982177 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73971be-21d1-4e2d-a927-8d542160457f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:40.983411 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f6b7ce-6b23-4f5e-a7ea-4ba5203a694e None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:40.984094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73971be-21d1-4e2d-a927-8d542160457f None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:40.991810 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:40.994123 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 175/348] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:49:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:49:40.995651 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453dfaa-96cf-4e9e-bd77-27a048a61010 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:40.996238 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453dfaa-96cf-4e9e-bd77-27a048a61010 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:40.996787 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453dfaa-96cf-4e9e-bd77-27a048a61010 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:40.998655 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 176/349] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:40.999639 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0453dfaa-96cf-4e9e-bd77-27a048a61010 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:41.020740 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25514ba4-3ab6-4553-8ea8-04dbe2f991b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:41.021581 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25514ba4-3ab6-4553-8ea8-04dbe2f991b7 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:41.021701 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25514ba4-3ab6-4553-8ea8-04dbe2f991b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:41.021945 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25514ba4-3ab6-4553-8ea8-04dbe2f991b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:41.022785 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25514ba4-3ab6-4553-8ea8-04dbe2f991b7 None admin] RBAC: Adding query filter params (group.id=e66391be7a2e4d2dab0bec9fcf9bdab7, role.id=c5431c9deadf47e496264ba5b78eeaf6, scope.project.id=b1e275c904b64687b29e7ca3d28fdda7) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:41.023096 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25514ba4-3ab6-4553-8ea8-04dbe2f991b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:41.025209 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25514ba4-3ab6-4553-8ea8-04dbe2f991b7 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:41.031543 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 174/350] 10.222.0.13 () {56 vars in 1376 bytes} [Tue Jun 7 21:49:40 2022] GET /identity/v3/role_assignments?group.id=e66391be7a2e4d2dab0bec9fcf9bdab7&scope.project.id=b1e275c904b64687b29e7ca3d28fdda7&role.id=c5431c9deadf47e496264ba5b78eeaf6 => generated 260 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:41.032427 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25514ba4-3ab6-4553-8ea8-04dbe2f991b7 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:41.124261 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0453dfaa-96cf-4e9e-bd77-27a048a61010 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:41.146226 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25514ba4-3ab6-4553-8ea8-04dbe2f991b7 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:41.155923 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:41.167588 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d79c671-a89e-412d-b4f2-29b1d7d0b1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:41.169167 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79c671-a89e-412d-b4f2-29b1d7d0b1e2 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:41.169681 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79c671-a89e-412d-b4f2-29b1d7d0b1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:41.170231 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79c671-a89e-412d-b4f2-29b1d7d0b1e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:41.171351 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d79c671-a89e-412d-b4f2-29b1d7d0b1e2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:41.171882 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d79c671-a89e-412d-b4f2-29b1d7d0b1e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:41.173524 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d79c671-a89e-412d-b4f2-29b1d7d0b1e2 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:41.185211 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 177/351] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jun 7 21:49:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:41.186916 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ac647f-fcaf-46ec-822e-41c5c8a742fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:41.187283 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd8f32-e800-48b2-aa13-67f6de29c036 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:41.188586 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac647f-fcaf-46ec-822e-41c5c8a742fc None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:41.188586 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac647f-fcaf-46ec-822e-41c5c8a742fc None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:41.188586 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac647f-fcaf-46ec-822e-41c5c8a742fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:41.189267 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd8f32-e800-48b2-aa13-67f6de29c036 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:41.189817 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd8f32-e800-48b2-aa13-67f6de29c036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:41.213158 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-6ffd8f32-e800-48b2-aa13-67f6de29c036 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:41.214794 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-6ffd8f32-e800-48b2-aa13-67f6de29c036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:41.217630 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:41.275772 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ac647f-fcaf-46ec-822e-41c5c8a742fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:41.277246 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ac647f-fcaf-46ec-822e-41c5c8a742fc None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:41.296618 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 178/352] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:49:41.298299 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb882e-57f6-4260-8c0e-4e947bcba86d None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:41.298854 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb882e-57f6-4260-8c0e-4e947bcba86d None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:41.299352 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb882e-57f6-4260-8c0e-4e947bcba86d None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:41.301292 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 179/353] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:41.303340 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efb882e-57f6-4260-8c0e-4e947bcba86d None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:41.414670 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 175/354] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:49:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 21:49:41.417253 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0580329-651b-4710-8740-8d683bdafe98 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:41.418069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0580329-651b-4710-8740-8d683bdafe98 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:41.418535 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0580329-651b-4710-8740-8d683bdafe98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:41.440821 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-b0580329-651b-4710-8740-8d683bdafe98 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:41.442830 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-b0580329-651b-4710-8740-8d683bdafe98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:41.444627 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efb882e-57f6-4260-8c0e-4e947bcba86d None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:41.474320 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:41.497422 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12da8912-3c6f-4e79-bab5-37e62984373a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:41.498053 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:41.499856 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da8912-3c6f-4e79-bab5-37e62984373a None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:41.500559 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da8912-3c6f-4e79-bab5-37e62984373a None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:41.501029 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da8912-3c6f-4e79-bab5-37e62984373a None admin] PATH_INFO: `/v3/users/1f699db471e64cc6ad48d6410e36184b` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:41.502534 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12da8912-3c6f-4e79-bab5-37e62984373a None admin] RBAC: Authorizing `identity:get_user(user_id=1f699db471e64cc6ad48d6410e36184b)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:41.504432 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12da8912-3c6f-4e79-bab5-37e62984373a None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:41.507245 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 180/355] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:49:41 2022] GET /identity/v3/users/1f699db471e64cc6ad48d6410e36184b => generated 293 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:41.512432 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12da8912-3c6f-4e79-bab5-37e62984373a None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:41.532546 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 176/356] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:49:41.535765 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0580329-651b-4710-8740-8d683bdafe98 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:41.660302 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12da8912-3c6f-4e79-bab5-37e62984373a None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:41.672394 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0580329-651b-4710-8740-8d683bdafe98 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:41.706898 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d21e7bb-bbba-430f-9455-d19761f5d57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:41.707854 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21e7bb-bbba-430f-9455-d19761f5d57c None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:41.708139 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21e7bb-bbba-430f-9455-d19761f5d57c None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:41.708418 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21e7bb-bbba-430f-9455-d19761f5d57c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:41.711435 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ff368-0ce9-41db-8f2f-fef88e1986af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:41.712502 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ff368-0ce9-41db-8f2f-fef88e1986af None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:41.712970 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ff368-0ce9-41db-8f2f-fef88e1986af None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:41.713422 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ff368-0ce9-41db-8f2f-fef88e1986af None admin] PATH_INFO: `/v3/projects/b1e275c904b64687b29e7ca3d28fdda7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:41.714801 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9ff368-0ce9-41db-8f2f-fef88e1986af None admin] RBAC: Authorizing `identity:get_project(project_id=b1e275c904b64687b29e7ca3d28fdda7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:41.716480 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9ff368-0ce9-41db-8f2f-fef88e1986af None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:41.718755 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 177/357] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:49:41 2022] GET /identity/v3/projects/b1e275c904b64687b29e7ca3d28fdda7 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:41.719770 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ff368-0ce9-41db-8f2f-fef88e1986af None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:41.722733 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d21e7bb-bbba-430f-9455-d19761f5d57c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:41.730796 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-9d21e7bb-bbba-430f-9455-d19761f5d57c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 21:49:41.730796 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:41.730796 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:41.730796 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:41.730796 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:41.730796 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:41.730796 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:41.730796 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:41.730796 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:41.730796 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:41.730796 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:41.730796 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:41.730796 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:41.730796 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:41.730796 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:41.730796 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:41.731769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:41.731769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:41.731769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:41.731769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:41.731769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:41.731769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:41.731769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:41.731769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:41.731769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:41.731769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:41.731769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:41.731769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:41.731769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:41.731769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:41.731769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:41.731769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:41.732578 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:41.732578 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:41.732578 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:41.732578 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:41.732578 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:41.732578 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:41.732578 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:41.732578 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:41.732578 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:41.732578 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:41.732578 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 21:49:41.732578 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:49:41.732578 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 181/358] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:49:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:49:41.733249 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d21e7bb-bbba-430f-9455-d19761f5d57c None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:41.770747 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:41.787638 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:41.843704 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ff368-0ce9-41db-8f2f-fef88e1986af None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:41.850971 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d21e7bb-bbba-430f-9455-d19761f5d57c None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:41.895100 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2858a979-00b2-450e-b8e2-afa97446ca40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:41.895468 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7397c52c-ae66-4636-9088-b8554a08f46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:41.895942 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2858a979-00b2-450e-b8e2-afa97446ca40 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:41.896066 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7397c52c-ae66-4636-9088-b8554a08f46b None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:41.896219 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2858a979-00b2-450e-b8e2-afa97446ca40 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:41.896308 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7397c52c-ae66-4636-9088-b8554a08f46b None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:41.896495 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2858a979-00b2-450e-b8e2-afa97446ca40 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:41.896586 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7397c52c-ae66-4636-9088-b8554a08f46b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:41.897396 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7397c52c-ae66-4636-9088-b8554a08f46b None admin] RBAC: Adding query filter params (role.id=30b0c2ba85ff481eb256122140e4163a, user.id=1f699db471e64cc6ad48d6410e36184b, scope.system=all) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:41.897709 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7397c52c-ae66-4636-9088-b8554a08f46b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:41.899480 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7397c52c-ae66-4636-9088-b8554a08f46b None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:41.910230 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2858a979-00b2-450e-b8e2-afa97446ca40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:41.918394 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 178/359] 10.222.0.13 () {56 vars in 1308 bytes} [Tue Jun 7 21:49:41 2022] GET /identity/v3/role_assignments?user.id=1f699db471e64cc6ad48d6410e36184b&scope.system=all&role.id=30b0c2ba85ff481eb256122140e4163a => generated 510 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:41.919382 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-2858a979-00b2-450e-b8e2-afa97446ca40 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 21:49:41.919382 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:41.919382 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:41.919382 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:41.919382 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:41.919382 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:41.919382 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:41.919382 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:41.919382 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:41.919382 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:41.919382 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:41.919382 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:41.919382 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:41.919382 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:41.919382 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:41.919382 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:41.920272 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:41.920272 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:41.920272 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:41.920272 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:41.920272 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:41.920272 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:41.920272 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:41.920272 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:41.920272 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:41.920272 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:41.920272 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:41.920272 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:41.920272 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:41.920272 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:41.920272 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:41.920272 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:41.921073 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:41.921073 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:41.921073 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:41.921073 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:41.921073 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:41.921073 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:41.921073 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:41.921073 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:41.921073 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:41.921073 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:41.921073 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 21:49:41.921073 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:49:41.921073 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 182/360] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Jun 7 21:49:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:49:41.921759 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2858a979-00b2-450e-b8e2-afa97446ca40 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:41.923008 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7397c52c-ae66-4636-9088-b8554a08f46b None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:42.052149 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2858a979-00b2-450e-b8e2-afa97446ca40 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:42.064444 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:42.077637 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:42.085497 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7397c52c-ae66-4636-9088-b8554a08f46b None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:42.112582 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf3a906-6492-44c3-9944-ec8f72af34ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:42.113407 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf3a906-6492-44c3-9944-ec8f72af34ea None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:42.113697 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf3a906-6492-44c3-9944-ec8f72af34ea None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:42.114045 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf3a906-6492-44c3-9944-ec8f72af34ea None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:42.114798 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf3a906-6492-44c3-9944-ec8f72af34ea None admin] RBAC: Adding query filter params (name=member) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:42.115112 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf3a906-6492-44c3-9944-ec8f72af34ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:42.117052 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf3a906-6492-44c3-9944-ec8f72af34ea None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:42.129796 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 183/361] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jun 7 21:49:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:42.131870 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef986f6-8e92-4aae-8c77-ae2f13fd8b8d None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:42.132393 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef986f6-8e92-4aae-8c77-ae2f13fd8b8d None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:42.133236 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef986f6-8e92-4aae-8c77-ae2f13fd8b8d None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:42.136403 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 184/362] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:42.137325 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef986f6-8e92-4aae-8c77-ae2f13fd8b8d None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:42.140450 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9947fc-0e44-4dc0-8702-22403709611f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:42.141738 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9947fc-0e44-4dc0-8702-22403709611f None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:42.142507 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9947fc-0e44-4dc0-8702-22403709611f None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:42.143271 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9947fc-0e44-4dc0-8702-22403709611f None admin] PATH_INFO: `/v3/groups/e66391be7a2e4d2dab0bec9fcf9bdab7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:42.145397 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9947fc-0e44-4dc0-8702-22403709611f None admin] RBAC: Authorizing `identity:get_group(group_id=e66391be7a2e4d2dab0bec9fcf9bdab7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:42.147067 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9947fc-0e44-4dc0-8702-22403709611f None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:42.149919 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 179/363] 10.222.0.13 () {56 vars in 1180 bytes} [Tue Jun 7 21:49:41 2022] GET /identity/v3/groups/e66391be7a2e4d2dab0bec9fcf9bdab7 => generated 226 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:42.150890 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9947fc-0e44-4dc0-8702-22403709611f None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:42.289473 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9947fc-0e44-4dc0-8702-22403709611f None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:42.307730 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef986f6-8e92-4aae-8c77-ae2f13fd8b8d None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:42.334372 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f77245-2a1c-4561-8b11-d32807207ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:42.335025 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f77245-2a1c-4561-8b11-d32807207ccb None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:42.335315 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f77245-2a1c-4561-8b11-d32807207ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:42.335618 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f77245-2a1c-4561-8b11-d32807207ccb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:42.353838 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc18993-7c67-4de4-978e-9bb6820e5352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:42.354524 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc18993-7c67-4de4-978e-9bb6820e5352 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:42.354810 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc18993-7c67-4de4-978e-9bb6820e5352 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:42.355195 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc18993-7c67-4de4-978e-9bb6820e5352 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:42.355941 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc18993-7c67-4de4-978e-9bb6820e5352 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:42.356250 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc18993-7c67-4de4-978e-9bb6820e5352 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:42.357627 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc18993-7c67-4de4-978e-9bb6820e5352 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:42.370043 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 185/364] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:49:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:42.371174 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc18993-7c67-4de4-978e-9bb6820e5352 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:42.383915 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:42.416270 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:42.433957 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f77245-2a1c-4561-8b11-d32807207ccb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:42.436510 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f77245-2a1c-4561-8b11-d32807207ccb None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:42.504791 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc18993-7c67-4de4-978e-9bb6820e5352 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:42.563241 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0006d2b6-e6d3-4f3a-92c6-0a8eac840953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:42.563879 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0006d2b6-e6d3-4f3a-92c6-0a8eac840953 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:42.564169 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0006d2b6-e6d3-4f3a-92c6-0a8eac840953 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:42.564454 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0006d2b6-e6d3-4f3a-92c6-0a8eac840953 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:42.565295 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0006d2b6-e6d3-4f3a-92c6-0a8eac840953 None admin] RBAC: Adding query filter params (group.id=e66391be7a2e4d2dab0bec9fcf9bdab7, role.id=50e04e9d09e045b0bcf8e1782a6e2f23, scope.project.id=b1e275c904b64687b29e7ca3d28fdda7) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:42.565612 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0006d2b6-e6d3-4f3a-92c6-0a8eac840953 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:42.567089 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0006d2b6-e6d3-4f3a-92c6-0a8eac840953 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:42.572622 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 180/365] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:49:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 21:49:42.574188 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b525f1-9397-4ca4-90e1-f824a40b1cd2 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:42.574297 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 186/366] 10.222.0.13 () {56 vars in 1376 bytes} [Tue Jun 7 21:49:42 2022] GET /identity/v3/role_assignments?group.id=e66391be7a2e4d2dab0bec9fcf9bdab7&scope.project.id=b1e275c904b64687b29e7ca3d28fdda7&role.id=50e04e9d09e045b0bcf8e1782a6e2f23 => generated 260 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:42.574814 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b525f1-9397-4ca4-90e1-f824a40b1cd2 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:42.575099 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0006d2b6-e6d3-4f3a-92c6-0a8eac840953 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:42.575723 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b525f1-9397-4ca4-90e1-f824a40b1cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:42.596570 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-60b525f1-9397-4ca4-90e1-f824a40b1cd2 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:42.599085 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-60b525f1-9397-4ca4-90e1-f824a40b1cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:42.677745 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:42.687708 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 181/367] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:49:42.688912 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b525f1-9397-4ca4-90e1-f824a40b1cd2 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:42.719091 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:42.740242 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0006d2b6-e6d3-4f3a-92c6-0a8eac840953 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:42.787128 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2e88bc-d319-4a4e-86b9-2b9426a6cca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:42.788225 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2e88bc-d319-4a4e-86b9-2b9426a6cca3 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:42.788719 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2e88bc-d319-4a4e-86b9-2b9426a6cca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:42.789201 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2e88bc-d319-4a4e-86b9-2b9426a6cca3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:42.828253 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b525f1-9397-4ca4-90e1-f824a40b1cd2 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:42.865616 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446a816e-8935-4bdd-87ee-88296b6a82fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:42.867095 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a816e-8935-4bdd-87ee-88296b6a82fd None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:42.867413 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a816e-8935-4bdd-87ee-88296b6a82fd None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:42.867696 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a816e-8935-4bdd-87ee-88296b6a82fd None admin] PATH_INFO: `/v3/projects/e8ca129f01d149b5aab391353f90139b` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:42.869198 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446a816e-8935-4bdd-87ee-88296b6a82fd None admin] RBAC: Authorizing `identity:get_project(project_id=e8ca129f01d149b5aab391353f90139b)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:42.869687 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2e88bc-d319-4a4e-86b9-2b9426a6cca3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:42.871230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446a816e-8935-4bdd-87ee-88296b6a82fd None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:42.872182 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2e88bc-d319-4a4e-86b9-2b9426a6cca3 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:42.872481 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 182/368] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:49:42 2022] GET /identity/v3/projects/e8ca129f01d149b5aab391353f90139b => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:42.873289 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446a816e-8935-4bdd-87ee-88296b6a82fd None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:42.950205 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:42.988617 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:42.993874 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446a816e-8935-4bdd-87ee-88296b6a82fd None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:42.997180 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 187/369] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:49:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 21:49:42.998344 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2e88bc-d319-4a4e-86b9-2b9426a6cca3 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:43.035975 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e68b3ee-ed16-4fbd-a45b-498e7a8d4e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:43.036784 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e68b3ee-ed16-4fbd-a45b-498e7a8d4e14 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:43.037070 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e68b3ee-ed16-4fbd-a45b-498e7a8d4e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:43.037349 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e68b3ee-ed16-4fbd-a45b-498e7a8d4e14 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:43.049132 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e68b3ee-ed16-4fbd-a45b-498e7a8d4e14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:43.059246 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-2e68b3ee-ed16-4fbd-a45b-498e7a8d4e14 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:49:43.059246 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:43.059246 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:43.059246 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:43.059246 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:43.059246 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:43.059246 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:43.059246 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:43.059246 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:43.059246 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:43.059246 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:43.059246 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:43.059246 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:43.059246 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:43.059246 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:43.059246 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:43.060234 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:43.060234 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:43.060234 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:43.060234 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:43.060234 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:43.060234 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:43.060234 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:43.060234 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:43.060234 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:43.060234 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:43.060234 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:43.060234 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:43.060234 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:43.060234 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:43.060234 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:43.060234 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:43.061034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:43.061034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:43.061034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:43.061034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:43.061034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:43.061034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:43.061034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:43.061034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:43.061034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:43.061034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:43.061034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:49:43.061034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:49:43.061034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 183/370] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jun 7 21:49:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:49:43.061702 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e68b3ee-ed16-4fbd-a45b-498e7a8d4e14 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:43.113962 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2e88bc-d319-4a4e-86b9-2b9426a6cca3 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:43.150270 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d70526-6d19-4f94-ba72-af95d3e34dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:43.151575 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d70526-6d19-4f94-ba72-af95d3e34dcb None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:43.152064 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d70526-6d19-4f94-ba72-af95d3e34dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:43.152568 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d70526-6d19-4f94-ba72-af95d3e34dcb None admin] PATH_INFO: `/v3/groups/e66391be7a2e4d2dab0bec9fcf9bdab7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:43.153970 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d70526-6d19-4f94-ba72-af95d3e34dcb None admin] RBAC: Authorizing `identity:get_group(group_id=e66391be7a2e4d2dab0bec9fcf9bdab7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:43.155776 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d70526-6d19-4f94-ba72-af95d3e34dcb None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:43.157687 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 188/371] 10.222.0.13 () {56 vars in 1180 bytes} [Tue Jun 7 21:49:42 2022] GET /identity/v3/groups/e66391be7a2e4d2dab0bec9fcf9bdab7 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:43.158739 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d70526-6d19-4f94-ba72-af95d3e34dcb None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:43.185625 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e68b3ee-ed16-4fbd-a45b-498e7a8d4e14 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:43.211540 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:43.225200 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6924bc-d11f-4d52-b30c-96f89e91796c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:43.226341 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6924bc-d11f-4d52-b30c-96f89e91796c None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:43.226632 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6924bc-d11f-4d52-b30c-96f89e91796c None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:43.229175 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6924bc-d11f-4d52-b30c-96f89e91796c None admin] PATH_INFO: `/v3/projects/e8ca129f01d149b5aab391353f90139b` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:43.230578 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6924bc-d11f-4d52-b30c-96f89e91796c None admin] RBAC: Authorizing `identity:get_project(project_id=e8ca129f01d149b5aab391353f90139b)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:43.232078 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6924bc-d11f-4d52-b30c-96f89e91796c None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:43.234249 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 184/372] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:49:43 2022] GET /identity/v3/projects/e8ca129f01d149b5aab391353f90139b => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:43.235129 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6924bc-d11f-4d52-b30c-96f89e91796c None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:43.247556 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:43.303687 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d70526-6d19-4f94-ba72-af95d3e34dcb None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:43.349947 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b575d8-095a-4b7f-b1b4-6cf17ba2a0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:43.350694 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b575d8-095a-4b7f-b1b4-6cf17ba2a0ab None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:43.351098 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b575d8-095a-4b7f-b1b4-6cf17ba2a0ab None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:43.351436 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b575d8-095a-4b7f-b1b4-6cf17ba2a0ab None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:43.352322 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b575d8-095a-4b7f-b1b4-6cf17ba2a0ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:43.352629 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b575d8-095a-4b7f-b1b4-6cf17ba2a0ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:43.354017 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b575d8-095a-4b7f-b1b4-6cf17ba2a0ab None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:43.365347 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 189/373] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:49:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:43.366211 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b575d8-095a-4b7f-b1b4-6cf17ba2a0ab None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:43.387620 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6924bc-d11f-4d52-b30c-96f89e91796c None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:43.440348 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a61dcc-849a-482e-a291-67b000bf0e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:43.442332 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a61dcc-849a-482e-a291-67b000bf0e7d None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:43.442623 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a61dcc-849a-482e-a291-67b000bf0e7d None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:43.442903 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a61dcc-849a-482e-a291-67b000bf0e7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:43.444092 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a61dcc-849a-482e-a291-67b000bf0e7d None admin] RBAC: Adding query filter params (group.id=e66391be7a2e4d2dab0bec9fcf9bdab7, role.id=c5431c9deadf47e496264ba5b78eeaf6, scope.project.id=e8ca129f01d149b5aab391353f90139b) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:43.444441 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a61dcc-849a-482e-a291-67b000bf0e7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:43.446800 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a61dcc-849a-482e-a291-67b000bf0e7d None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:43.459353 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 185/374] 10.222.0.13 () {56 vars in 1376 bytes} [Tue Jun 7 21:49:43 2022] GET /identity/v3/role_assignments?group.id=e66391be7a2e4d2dab0bec9fcf9bdab7&scope.project.id=e8ca129f01d149b5aab391353f90139b&role.id=c5431c9deadf47e496264ba5b78eeaf6 => generated 260 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:43.459353 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a61dcc-849a-482e-a291-67b000bf0e7d None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:43.555374 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b575d8-095a-4b7f-b1b4-6cf17ba2a0ab None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:43.565469 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:43.620890 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:43.649632 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a61dcc-849a-482e-a291-67b000bf0e7d None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:43.653652 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ecd19-b9de-497c-84cf-108a7af6bb1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:43.654540 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ecd19-b9de-497c-84cf-108a7af6bb1e None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:43.654824 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ecd19-b9de-497c-84cf-108a7af6bb1e None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:43.655102 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ecd19-b9de-497c-84cf-108a7af6bb1e None admin] PATH_INFO: `/v3/groups/e66391be7a2e4d2dab0bec9fcf9bdab7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:43.656383 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882ecd19-b9de-497c-84cf-108a7af6bb1e None admin] RBAC: Authorizing `identity:get_group(group_id=e66391be7a2e4d2dab0bec9fcf9bdab7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:43.659172 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882ecd19-b9de-497c-84cf-108a7af6bb1e None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:43.666974 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 190/375] 10.222.0.13 () {56 vars in 1180 bytes} [Tue Jun 7 21:49:43 2022] GET /identity/v3/groups/e66391be7a2e4d2dab0bec9fcf9bdab7 => generated 226 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:43.667969 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ecd19-b9de-497c-84cf-108a7af6bb1e None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:43.732619 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51c81bb-5ef0-4b65-9d41-732ab3787a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:43.733283 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c81bb-5ef0-4b65-9d41-732ab3787a83 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:43.733566 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c81bb-5ef0-4b65-9d41-732ab3787a83 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:43.733842 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c81bb-5ef0-4b65-9d41-732ab3787a83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:43.891442 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ecd19-b9de-497c-84cf-108a7af6bb1e None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:43.908329 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51c81bb-5ef0-4b65-9d41-732ab3787a83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:43.910558 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51c81bb-5ef0-4b65-9d41-732ab3787a83 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:43.940486 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fe9900-6672-4484-afd7-9b2d7656c55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:43.941242 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fe9900-6672-4484-afd7-9b2d7656c55d None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:43.941643 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fe9900-6672-4484-afd7-9b2d7656c55d None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:43.942075 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fe9900-6672-4484-afd7-9b2d7656c55d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:43.943005 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fe9900-6672-4484-afd7-9b2d7656c55d None admin] RBAC: Adding query filter params (group.id=e66391be7a2e4d2dab0bec9fcf9bdab7, role.id=50e04e9d09e045b0bcf8e1782a6e2f23, scope.project.id=e8ca129f01d149b5aab391353f90139b) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:43.943354 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fe9900-6672-4484-afd7-9b2d7656c55d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:43.945262 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fe9900-6672-4484-afd7-9b2d7656c55d None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:43.955942 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 191/376] 10.222.0.13 () {56 vars in 1376 bytes} [Tue Jun 7 21:49:43 2022] GET /identity/v3/role_assignments?group.id=e66391be7a2e4d2dab0bec9fcf9bdab7&scope.project.id=e8ca129f01d149b5aab391353f90139b&role.id=50e04e9d09e045b0bcf8e1782a6e2f23 => generated 260 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:43.958463 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21403c67-8c2b-49aa-98ad-89be37478712 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:43.958890 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21403c67-8c2b-49aa-98ad-89be37478712 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:43.959309 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21403c67-8c2b-49aa-98ad-89be37478712 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:43.964176 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 192/377] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:43.974170 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa77dc-6e18-4622-a515-7bfbd7cda0a7 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:43.979364 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa77dc-6e18-4622-a515-7bfbd7cda0a7 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:43.979364 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa77dc-6e18-4622-a515-7bfbd7cda0a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:44.015721 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-8dfa77dc-6e18-4622-a515-7bfbd7cda0a7 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:44.018634 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-8dfa77dc-6e18-4622-a515-7bfbd7cda0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:44.063372 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:44.108767 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:44.143438 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 193/378] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:49:44.155349 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 186/379] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:49:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 697 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 21:49:44.161882 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfa77dc-6e18-4622-a515-7bfbd7cda0a7 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:44.171374 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51c81bb-5ef0-4b65-9d41-732ab3787a83 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:44.328618 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51c81bb-5ef0-4b65-9d41-732ab3787a83 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:44.361405 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfa77dc-6e18-4622-a515-7bfbd7cda0a7 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:44.378733 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e165c954-c421-4df5-b8e4-9a98f778c1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:44.379595 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e165c954-c421-4df5-b8e4-9a98f778c1c6 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:44.379891 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e165c954-c421-4df5-b8e4-9a98f778c1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:44.380191 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e165c954-c421-4df5-b8e4-9a98f778c1c6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:44.396037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e165c954-c421-4df5-b8e4-9a98f778c1c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:44.410955 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b4c42-b15e-4539-a094-235bf9939963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:44.411198 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-e165c954-c421-4df5-b8e4-9a98f778c1c6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 21:49:44.411198 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:44.411198 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:44.411198 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:44.411198 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:44.411198 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:44.411198 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:44.411198 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:44.411198 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:44.411198 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:44.411198 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:44.411198 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:44.411198 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:44.411198 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:44.411198 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:44.411198 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:44.412065 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:44.412065 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:44.412065 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:44.412065 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:44.412065 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:44.412065 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:44.412065 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:44.412065 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:44.412065 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:44.412065 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:44.412065 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:44.412065 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:44.412065 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:44.412065 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:44.412065 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:44.412065 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:44.412881 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:44.412881 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:44.412881 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:44.412881 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:44.412881 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:44.412881 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:44.412881 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:44.412881 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:44.412881 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:44.412881 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:44.412881 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 21:49:44.412881 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:49:44.413493 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b4c42-b15e-4539-a094-235bf9939963 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:44.413493 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b4c42-b15e-4539-a094-235bf9939963 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:44.413493 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b4c42-b15e-4539-a094-235bf9939963 None admin] PATH_INFO: `/v3/projects/c61211fad6e447b0bf162f99f1111a82` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:44.413685 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 187/380] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:49:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:49:44.414359 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82b4c42-b15e-4539-a094-235bf9939963 None admin] RBAC: Authorizing `identity:get_project(project_id=c61211fad6e447b0bf162f99f1111a82)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:44.417148 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82b4c42-b15e-4539-a094-235bf9939963 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:44.419243 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 194/381] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:49:44 2022] GET /identity/v3/projects/c61211fad6e447b0bf162f99f1111a82 => generated 344 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:44.422861 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e165c954-c421-4df5-b8e4-9a98f778c1c6 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:44.424562 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b4c42-b15e-4539-a094-235bf9939963 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:44.462820 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:44.543852 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:44.630405 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e165c954-c421-4df5-b8e4-9a98f778c1c6 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:44.677480 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b4c42-b15e-4539-a094-235bf9939963 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:44.681757 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b67688-70d6-4224-afb3-a48a07c18a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:44.682446 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b67688-70d6-4224-afb3-a48a07c18a9f None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:44.682731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b67688-70d6-4224-afb3-a48a07c18a9f None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:44.683008 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b67688-70d6-4224-afb3-a48a07c18a9f None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:44.684059 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b67688-70d6-4224-afb3-a48a07c18a9f None admin] RBAC: Adding query filter params (name=member) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:44.684369 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b67688-70d6-4224-afb3-a48a07c18a9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:44.685760 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b67688-70d6-4224-afb3-a48a07c18a9f None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:44.696781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 188/382] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jun 7 21:49:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:44.703494 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b67688-70d6-4224-afb3-a48a07c18a9f None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:44.754042 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e2e3a5-1285-41f8-a97c-e9cfac2ea31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:44.755444 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2e3a5-1285-41f8-a97c-e9cfac2ea31d None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:44.755752 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2e3a5-1285-41f8-a97c-e9cfac2ea31d None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:44.756035 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2e3a5-1285-41f8-a97c-e9cfac2ea31d None admin] PATH_INFO: `/v3/groups/c01abaa0eb6d4bd5aa43790d53db8d32` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:44.776948 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e2e3a5-1285-41f8-a97c-e9cfac2ea31d None admin] RBAC: Authorizing `identity:get_group(group_id=c01abaa0eb6d4bd5aa43790d53db8d32)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:44.778422 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e2e3a5-1285-41f8-a97c-e9cfac2ea31d None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:44.780375 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 195/383] 10.222.0.13 () {56 vars in 1180 bytes} [Tue Jun 7 21:49:44 2022] GET /identity/v3/groups/c01abaa0eb6d4bd5aa43790d53db8d32 => generated 229 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:49:44.798568 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e2e3a5-1285-41f8-a97c-e9cfac2ea31d None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:44.851711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:44.895040 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b67688-70d6-4224-afb3-a48a07c18a9f None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:44.963427 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ff1f5-3161-447d-a897-0819bc92ab9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:44.964215 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ff1f5-3161-447d-a897-0819bc92ab9c None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:44.964499 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ff1f5-3161-447d-a897-0819bc92ab9c None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:44.964778 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ff1f5-3161-447d-a897-0819bc92ab9c None admin] PATH_INFO: `/v3/groups/e66391be7a2e4d2dab0bec9fcf9bdab7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:44.968563 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050ff1f5-3161-447d-a897-0819bc92ab9c None admin] RBAC: Authorizing `identity:get_group(group_id=e66391be7a2e4d2dab0bec9fcf9bdab7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:44.970506 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050ff1f5-3161-447d-a897-0819bc92ab9c None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:44.974113 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 189/384] 10.222.0.13 () {56 vars in 1180 bytes} [Tue Jun 7 21:49:44 2022] GET /identity/v3/groups/e66391be7a2e4d2dab0bec9fcf9bdab7 => generated 226 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:44.980040 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ff1f5-3161-447d-a897-0819bc92ab9c None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:44.997465 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e2e3a5-1285-41f8-a97c-e9cfac2ea31d None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:45.043773 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:45.071777 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8016756-c60a-460b-aae1-e62c9a0e93bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:45.083450 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8016756-c60a-460b-aae1-e62c9a0e93bf None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:45.083450 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8016756-c60a-460b-aae1-e62c9a0e93bf None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:45.083450 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8016756-c60a-460b-aae1-e62c9a0e93bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:45.083450 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8016756-c60a-460b-aae1-e62c9a0e93bf None admin] RBAC: Adding query filter params (group.id=c01abaa0eb6d4bd5aa43790d53db8d32, role.id=6b483758dbd44eb4a78df2ce4603d601, scope.project.id=c61211fad6e447b0bf162f99f1111a82) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:45.083450 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8016756-c60a-460b-aae1-e62c9a0e93bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:45.084988 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8016756-c60a-460b-aae1-e62c9a0e93bf None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:45.098656 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 196/385] 10.222.0.13 () {56 vars in 1376 bytes} [Tue Jun 7 21:49:44 2022] GET /identity/v3/role_assignments?group.id=c01abaa0eb6d4bd5aa43790d53db8d32&scope.project.id=c61211fad6e447b0bf162f99f1111a82&role.id=6b483758dbd44eb4a78df2ce4603d601 => generated 260 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:49:45.142072 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ff1f5-3161-447d-a897-0819bc92ab9c None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:45.212217 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87857b7-d762-4aee-a0ac-946861f3e628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:45.213549 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87857b7-d762-4aee-a0ac-946861f3e628 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:45.214060 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87857b7-d762-4aee-a0ac-946861f3e628 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:45.214538 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87857b7-d762-4aee-a0ac-946861f3e628 None admin] PATH_INFO: `/v3/projects/b1e275c904b64687b29e7ca3d28fdda7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:45.216165 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87857b7-d762-4aee-a0ac-946861f3e628 None admin] RBAC: Authorizing `identity:get_project(project_id=b1e275c904b64687b29e7ca3d28fdda7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:45.217842 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87857b7-d762-4aee-a0ac-946861f3e628 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:45.220222 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 190/386] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:49:44 2022] GET /identity/v3/projects/b1e275c904b64687b29e7ca3d28fdda7 => generated 298 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:45.226722 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8016756-c60a-460b-aae1-e62c9a0e93bf None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:45.359877 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8016756-c60a-460b-aae1-e62c9a0e93bf None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:45.406783 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a406b-6f98-4874-979b-c75fb4dd6456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:45.407711 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a406b-6f98-4874-979b-c75fb4dd6456 None admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:45.407990 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a406b-6f98-4874-979b-c75fb4dd6456 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:45.408266 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a406b-6f98-4874-979b-c75fb4dd6456 None admin] PATH_INFO: `/v3/projects/b1e275c904b64687b29e7ca3d28fdda7/groups/e66391be7a2e4d2dab0bec9fcf9bdab7/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:45.412792 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9a406b-6f98-4874-979b-c75fb4dd6456 None admin] RBAC: Authorizing `identity:create_grant(project_id=b1e275c904b64687b29e7ca3d28fdda7, group_id=e66391be7a2e4d2dab0bec9fcf9bdab7, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:45.414258 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9a406b-6f98-4874-979b-c75fb4dd6456 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:45.435541 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 197/387] 10.222.0.13 () {58 vars in 1442 bytes} [Tue Jun 7 21:49:45 2022] PUT /identity/v3/projects/b1e275c904b64687b29e7ca3d28fdda7/groups/e66391be7a2e4d2dab0bec9fcf9bdab7/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 21:49:45.783183 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c1ee9-4200-4727-9f59-e92e3e5164dd None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:45.783719 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c1ee9-4200-4727-9f59-e92e3e5164dd None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:45.783792 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c1ee9-4200-4727-9f59-e92e3e5164dd None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:45.785472 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 191/388] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:45.791081 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be25109-4fc9-4bd4-91bb-64b7c520abe1 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:45.791405 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be25109-4fc9-4bd4-91bb-64b7c520abe1 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:45.791674 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be25109-4fc9-4bd4-91bb-64b7c520abe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:45.812369 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-5be25109-4fc9-4bd4-91bb-64b7c520abe1 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:45.813582 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-5be25109-4fc9-4bd4-91bb-64b7c520abe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:45.864155 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:45.899027 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 198/389] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:49:45.922314 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9c1ee9-4200-4727-9f59-e92e3e5164dd None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:45.940030 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:46.061553 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9c1ee9-4200-4727-9f59-e92e3e5164dd None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:46.106405 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e4585-5551-4a50-bd2d-c7480e67e6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:46.107687 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e4585-5551-4a50-bd2d-c7480e67e6e0 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:46.108206 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e4585-5551-4a50-bd2d-c7480e67e6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:46.108705 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e4585-5551-4a50-bd2d-c7480e67e6e0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:46.125466 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291e4585-5551-4a50-bd2d-c7480e67e6e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:46.134930 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-291e4585-5551-4a50-bd2d-c7480e67e6e0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 21:49:46.134930 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:46.134930 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:46.134930 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:46.134930 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:46.134930 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:46.134930 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:46.134930 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:46.134930 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:46.134930 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:46.134930 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:46.134930 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:46.134930 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:46.134930 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:46.134930 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:46.134930 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:46.140024 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:46.140024 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:46.140024 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:46.140024 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:46.140024 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:46.140024 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:46.140024 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:46.140024 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:46.140024 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:46.140024 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:46.140024 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:46.140024 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:46.140024 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:46.140024 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:46.140024 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:46.140024 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:46.140836 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:46.140836 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:46.140836 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:46.140836 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:46.140836 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:46.140836 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:46.140836 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:46.140836 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:46.140836 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:46.140836 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:46.140836 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 21:49:46.140836 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:49:46.141653 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 192/390] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Jun 7 21:49:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:49:46.148457 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be25109-4fc9-4bd4-91bb-64b7c520abe1 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:46.280858 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be25109-4fc9-4bd4-91bb-64b7c520abe1 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:46.322397 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec70120-c0e9-4d8d-8f2f-6130f7e63ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:46.322997 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec70120-c0e9-4d8d-8f2f-6130f7e63ed9 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:46.323279 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec70120-c0e9-4d8d-8f2f-6130f7e63ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:46.323689 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec70120-c0e9-4d8d-8f2f-6130f7e63ed9 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:46.324455 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec70120-c0e9-4d8d-8f2f-6130f7e63ed9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:46.324760 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec70120-c0e9-4d8d-8f2f-6130f7e63ed9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:46.326128 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec70120-c0e9-4d8d-8f2f-6130f7e63ed9 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:46.335120 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 199/391] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:49:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:46.340614 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e4585-5551-4a50-bd2d-c7480e67e6e0 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:46.449787 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:46.482347 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e4585-5551-4a50-bd2d-c7480e67e6e0 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:46.526203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e28ac3-8e6b-4f65-a403-bcde49e5f279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:46.527064 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e28ac3-8e6b-4f65-a403-bcde49e5f279 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:46.527367 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e28ac3-8e6b-4f65-a403-bcde49e5f279 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:46.527664 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e28ac3-8e6b-4f65-a403-bcde49e5f279 None admin] PATH_INFO: `/v3/groups/e66391be7a2e4d2dab0bec9fcf9bdab7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:46.530544 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e28ac3-8e6b-4f65-a403-bcde49e5f279 None admin] RBAC: Authorizing `identity:get_group(group_id=e66391be7a2e4d2dab0bec9fcf9bdab7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:46.532066 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e28ac3-8e6b-4f65-a403-bcde49e5f279 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:46.534101 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 193/392] 10.222.0.13 () {56 vars in 1180 bytes} [Tue Jun 7 21:49:46 2022] GET /identity/v3/groups/e66391be7a2e4d2dab0bec9fcf9bdab7 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:46.539883 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec70120-c0e9-4d8d-8f2f-6130f7e63ed9 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:46.574643 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a5f3d-b2ea-4d70-9996-6c142ad058c4 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:46.574825 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a5f3d-b2ea-4d70-9996-6c142ad058c4 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:46.575115 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a5f3d-b2ea-4d70-9996-6c142ad058c4 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:46.576854 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 194/393] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 21:49:46.583115 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f2e32-4fbe-4e3f-b052-c02954b9f36b None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:46.583426 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f2e32-4fbe-4e3f-b052-c02954b9f36b None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:46.583709 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f2e32-4fbe-4e3f-b052-c02954b9f36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:46.598941 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:46.606188 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-ea3f2e32-4fbe-4e3f-b052-c02954b9f36b None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:46.607680 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-ea3f2e32-4fbe-4e3f-b052-c02954b9f36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:46.687032 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec70120-c0e9-4d8d-8f2f-6130f7e63ed9 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:46.734229 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e8a4bf-9388-4ce7-a7b4-7e693d0adaf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:46.735107 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e8a4bf-9388-4ce7-a7b4-7e693d0adaf0 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:46.735414 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e8a4bf-9388-4ce7-a7b4-7e693d0adaf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:46.735699 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e8a4bf-9388-4ce7-a7b4-7e693d0adaf0 None admin] PATH_INFO: `/v3/projects/b1e275c904b64687b29e7ca3d28fdda7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:46.737620 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e8a4bf-9388-4ce7-a7b4-7e693d0adaf0 None admin] RBAC: Authorizing `identity:get_project(project_id=b1e275c904b64687b29e7ca3d28fdda7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:46.740342 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e8a4bf-9388-4ce7-a7b4-7e693d0adaf0 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:46.742292 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 200/394] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:49:46 2022] GET /identity/v3/projects/b1e275c904b64687b29e7ca3d28fdda7 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:46.748129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e8a4bf-9388-4ce7-a7b4-7e693d0adaf0 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:46.758012 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 195/395] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 21:49:46.784887 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3f2e32-4fbe-4e3f-b052-c02954b9f36b None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:46.922188 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:46.964375 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3f2e32-4fbe-4e3f-b052-c02954b9f36b None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:46.965896 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e8a4bf-9388-4ce7-a7b4-7e693d0adaf0 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:46.981936 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:47.017234 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f29da-34e8-4175-970c-e960c730af54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:47.018304 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f29da-34e8-4175-970c-e960c730af54 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:47.019481 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f29da-34e8-4175-970c-e960c730af54 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:47.020012 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f29da-34e8-4175-970c-e960c730af54 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:47.033597 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34f29da-34e8-4175-970c-e960c730af54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:47.043266 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-c34f29da-34e8-4175-970c-e960c730af54 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 21:49:47.043266 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:47.043266 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:47.043266 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:47.043266 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:47.043266 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:47.043266 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:47.043266 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:47.043266 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:47.043266 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:47.043266 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:47.043266 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:47.043266 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:47.043266 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:47.043266 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:47.043266 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:47.044378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:47.044378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:47.044378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:47.044378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:47.044378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:47.044378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:47.044378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:47.044378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:47.044378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:47.044378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:47.044378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:47.044378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:47.044378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:47.044378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:47.044378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:47.044378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:47.045427 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:47.045427 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:47.045427 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:47.045427 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:47.045427 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:47.045427 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:47.045427 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:47.045427 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:47.045427 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:47.045427 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:47.045427 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 21:49:47.045427 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:49:47.050034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 196/396] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:49:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:49:47.055796 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f29da-34e8-4175-970c-e960c730af54 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:47.064622 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad16b04b-f4e2-435a-bfd4-04de87c7d01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:47.065738 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad16b04b-f4e2-435a-bfd4-04de87c7d01d None admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:47.066124 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad16b04b-f4e2-435a-bfd4-04de87c7d01d None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:47.066506 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad16b04b-f4e2-435a-bfd4-04de87c7d01d None admin] PATH_INFO: `/v3/projects/b1e275c904b64687b29e7ca3d28fdda7/groups/e66391be7a2e4d2dab0bec9fcf9bdab7/roles/50e04e9d09e045b0bcf8e1782a6e2f23` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:47.073216 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad16b04b-f4e2-435a-bfd4-04de87c7d01d None admin] RBAC: Authorizing `identity:create_grant(project_id=b1e275c904b64687b29e7ca3d28fdda7, group_id=e66391be7a2e4d2dab0bec9fcf9bdab7, role_id=50e04e9d09e045b0bcf8e1782a6e2f23)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:47.074789 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad16b04b-f4e2-435a-bfd4-04de87c7d01d None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:47.111440 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 201/397] 10.222.0.13 () {58 vars in 1442 bytes} [Tue Jun 7 21:49:46 2022] PUT /identity/v3/projects/b1e275c904b64687b29e7ca3d28fdda7/groups/e66391be7a2e4d2dab0bec9fcf9bdab7/roles/50e04e9d09e045b0bcf8e1782a6e2f23 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 21:49:47.176430 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f29da-34e8-4175-970c-e960c730af54 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:47.221371 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:47.230765 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523711ec-a226-481f-9322-99686565314f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:47.232018 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523711ec-a226-481f-9322-99686565314f None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:47.232661 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523711ec-a226-481f-9322-99686565314f None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:47.232951 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523711ec-a226-481f-9322-99686565314f None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:47.234074 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523711ec-a226-481f-9322-99686565314f None admin] RBAC: Adding query filter params (name=member) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:47.234385 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523711ec-a226-481f-9322-99686565314f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:47.235855 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523711ec-a226-481f-9322-99686565314f None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:47.246455 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 197/398] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jun 7 21:49:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:47.252033 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad16b04b-f4e2-435a-bfd4-04de87c7d01d None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:47.334033 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95823eab-0144-46f3-819a-81d70a7f0359 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:47.334384 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95823eab-0144-46f3-819a-81d70a7f0359 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:47.334785 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95823eab-0144-46f3-819a-81d70a7f0359 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:47.336392 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 198/399] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:47.342691 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c5592-424d-455b-80a5-fd9ecfa88ab2 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:47.342985 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c5592-424d-455b-80a5-fd9ecfa88ab2 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:47.343855 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c5592-424d-455b-80a5-fd9ecfa88ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:47.380581 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-2a5c5592-424d-455b-80a5-fd9ecfa88ab2 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:47.381853 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-2a5c5592-424d-455b-80a5-fd9ecfa88ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:47.418287 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad16b04b-f4e2-435a-bfd4-04de87c7d01d None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:47.457281 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a713440-0d13-4004-98eb-706693121002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:47.458235 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a713440-0d13-4004-98eb-706693121002 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:47.458530 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a713440-0d13-4004-98eb-706693121002 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:47.458813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a713440-0d13-4004-98eb-706693121002 None admin] PATH_INFO: `/v3/groups/e66391be7a2e4d2dab0bec9fcf9bdab7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:47.460280 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a713440-0d13-4004-98eb-706693121002 None admin] RBAC: Authorizing `identity:get_group(group_id=e66391be7a2e4d2dab0bec9fcf9bdab7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:47.461883 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a713440-0d13-4004-98eb-706693121002 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:47.466487 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 202/400] 10.222.0.13 () {56 vars in 1180 bytes} [Tue Jun 7 21:49:47 2022] GET /identity/v3/groups/e66391be7a2e4d2dab0bec9fcf9bdab7 => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:47.470817 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a713440-0d13-4004-98eb-706693121002 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:47.483089 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 199/401] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 21:49:47.506697 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:47.508389 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5c5592-424d-455b-80a5-fd9ecfa88ab2 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:47.587357 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a713440-0d13-4004-98eb-706693121002 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:47.626182 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e2f97f-f250-4fda-adea-8005489f6283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:47.627720 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2f97f-f250-4fda-adea-8005489f6283 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:47.628721 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2f97f-f250-4fda-adea-8005489f6283 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:47.629173 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2f97f-f250-4fda-adea-8005489f6283 None admin] PATH_INFO: `/v3/projects/e8ca129f01d149b5aab391353f90139b` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:47.630651 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e2f97f-f250-4fda-adea-8005489f6283 None admin] RBAC: Authorizing `identity:get_project(project_id=e8ca129f01d149b5aab391353f90139b)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:47.632451 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e2f97f-f250-4fda-adea-8005489f6283 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:47.639377 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 203/402] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:49:47 2022] GET /identity/v3/projects/e8ca129f01d149b5aab391353f90139b => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:47.640132 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e2f97f-f250-4fda-adea-8005489f6283 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:47.697464 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5c5592-424d-455b-80a5-fd9ecfa88ab2 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:47.751543 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e2f97f-f250-4fda-adea-8005489f6283 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:47.760393 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:47.771057 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:47.792727 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6fcc74-2037-4d3b-8268-2e3aaaee73c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:47.795682 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fcc74-2037-4d3b-8268-2e3aaaee73c3 None admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:47.796563 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fcc74-2037-4d3b-8268-2e3aaaee73c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:47.797462 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fcc74-2037-4d3b-8268-2e3aaaee73c3 None admin] PATH_INFO: `/v3/projects/e8ca129f01d149b5aab391353f90139b/groups/e66391be7a2e4d2dab0bec9fcf9bdab7/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:47.800802 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6fcc74-2037-4d3b-8268-2e3aaaee73c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=e8ca129f01d149b5aab391353f90139b, group_id=e66391be7a2e4d2dab0bec9fcf9bdab7, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:47.801558 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8a71fc-81bf-4c62-abed-e7e9e2cc0af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:47.802857 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a71fc-81bf-4c62-abed-e7e9e2cc0af1 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:47.804555 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6fcc74-2037-4d3b-8268-2e3aaaee73c3 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:47.807795 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a71fc-81bf-4c62-abed-e7e9e2cc0af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:47.808101 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a71fc-81bf-4c62-abed-e7e9e2cc0af1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:47.826618 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 204/403] 10.222.0.13 () {58 vars in 1442 bytes} [Tue Jun 7 21:49:47 2022] PUT /identity/v3/projects/e8ca129f01d149b5aab391353f90139b/groups/e66391be7a2e4d2dab0bec9fcf9bdab7/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 21:49:47.835194 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8a71fc-81bf-4c62-abed-e7e9e2cc0af1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:47.845401 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-ee8a71fc-81bf-4c62-abed-e7e9e2cc0af1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 21:49:47.845401 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:47.845401 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:47.845401 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:47.845401 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:47.845401 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:47.845401 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:47.845401 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:47.845401 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:47.845401 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:47.845401 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:47.845401 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:47.845401 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:47.845401 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:47.845401 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:47.845401 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:47.846491 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:47.846491 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:47.846491 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:47.846491 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:47.846491 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:47.846491 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:47.846491 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:47.846491 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:47.846491 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:47.846491 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:47.846491 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:47.846491 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:47.846491 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:47.846491 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:47.846491 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:47.846491 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:47.848037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:47.848037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:47.848037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:47.848037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:47.848037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:47.848037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:47.848037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:47.848037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:47.848037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:47.848037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:47.848037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 21:49:47.848037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:49:47.855686 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 200/404] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Jun 7 21:49:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:49:47.861753 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6fcc74-2037-4d3b-8268-2e3aaaee73c3 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:47.929190 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b04ddf-ac8e-479b-9bd9-66edd7c03310 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:47.929477 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b04ddf-ac8e-479b-9bd9-66edd7c03310 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:47.929759 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b04ddf-ac8e-479b-9bd9-66edd7c03310 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:47.933014 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 201/405] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:47.940972 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2ffbc-af0c-4594-92c4-52f8e667350b None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:47.941259 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2ffbc-af0c-4594-92c4-52f8e667350b None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:47.941539 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2ffbc-af0c-4594-92c4-52f8e667350b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:47.984596 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-55d2ffbc-af0c-4594-92c4-52f8e667350b None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:47.985830 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-55d2ffbc-af0c-4594-92c4-52f8e667350b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:48.003056 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6fcc74-2037-4d3b-8268-2e3aaaee73c3 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:48.044974 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de769eea-dc4c-492d-9c96-9628eba0e72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:48.046545 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de769eea-dc4c-492d-9c96-9628eba0e72d None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:48.047278 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de769eea-dc4c-492d-9c96-9628eba0e72d None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:48.047809 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de769eea-dc4c-492d-9c96-9628eba0e72d None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:48.048801 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de769eea-dc4c-492d-9c96-9628eba0e72d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:48.049313 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de769eea-dc4c-492d-9c96-9628eba0e72d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:48.051512 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de769eea-dc4c-492d-9c96-9628eba0e72d None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:48.062260 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 205/406] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:49:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:48.068455 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de769eea-dc4c-492d-9c96-9628eba0e72d None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:48.099760 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:48.113233 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 202/407] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:49:48.137541 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d2ffbc-af0c-4594-92c4-52f8e667350b None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:48.185947 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de769eea-dc4c-492d-9c96-9628eba0e72d None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:48.224987 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1864666e-6f00-48f7-8fdf-3ed09e081016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:48.226122 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1864666e-6f00-48f7-8fdf-3ed09e081016 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:48.226559 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1864666e-6f00-48f7-8fdf-3ed09e081016 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:48.226984 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1864666e-6f00-48f7-8fdf-3ed09e081016 None admin] PATH_INFO: `/v3/groups/e66391be7a2e4d2dab0bec9fcf9bdab7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:48.228467 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1864666e-6f00-48f7-8fdf-3ed09e081016 None admin] RBAC: Authorizing `identity:get_group(group_id=e66391be7a2e4d2dab0bec9fcf9bdab7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:48.230166 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1864666e-6f00-48f7-8fdf-3ed09e081016 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:48.232187 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 206/408] 10.222.0.13 () {56 vars in 1180 bytes} [Tue Jun 7 21:49:48 2022] GET /identity/v3/groups/e66391be7a2e4d2dab0bec9fcf9bdab7 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:49:48.237804 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1864666e-6f00-48f7-8fdf-3ed09e081016 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:48.248073 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d2ffbc-af0c-4594-92c4-52f8e667350b None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:48.268565 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:48.307444 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ae8e0-7644-4f2b-871c-71474c654157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:48.308397 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ae8e0-7644-4f2b-871c-71474c654157 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:48.308754 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ae8e0-7644-4f2b-871c-71474c654157 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:48.309103 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ae8e0-7644-4f2b-871c-71474c654157 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:48.332827 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61ae8e0-7644-4f2b-871c-71474c654157 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:48.343770 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:48.350231 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-a61ae8e0-7644-4f2b-871c-71474c654157 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:49:48.350231 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:48.350231 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:48.350231 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:48.350231 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:48.350231 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:48.350231 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:48.350231 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:48.350231 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:48.350231 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:48.350231 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:48.350231 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:48.350231 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:48.350231 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:48.350231 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:48.350231 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:48.351158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:48.351158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:48.351158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:48.351158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:48.351158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:48.351158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:48.351158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:48.351158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:48.351158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:48.351158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:48.351158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:48.351158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:48.351158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:48.351158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:48.351158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:48.351158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:48.352322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:48.352322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:48.352322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:48.352322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:48.352322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:48.352322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:48.352322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:48.352322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:48.352322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:48.352322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:48.352322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:49:48.352322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:49:48.352322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 203/409] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jun 7 21:49:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:49:48.353722 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1864666e-6f00-48f7-8fdf-3ed09e081016 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:48.357415 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ae8e0-7644-4f2b-871c-71474c654157 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:48.398504 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8040b3a-e184-4891-9e6c-173e417ed3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:48.399797 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8040b3a-e184-4891-9e6c-173e417ed3ee None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:48.400290 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8040b3a-e184-4891-9e6c-173e417ed3ee None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:48.401761 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8040b3a-e184-4891-9e6c-173e417ed3ee None admin] PATH_INFO: `/v3/projects/e8ca129f01d149b5aab391353f90139b` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:48.404474 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8040b3a-e184-4891-9e6c-173e417ed3ee None admin] RBAC: Authorizing `identity:get_project(project_id=e8ca129f01d149b5aab391353f90139b)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:48.406156 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8040b3a-e184-4891-9e6c-173e417ed3ee None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:48.408295 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 207/410] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:49:48 2022] GET /identity/v3/projects/e8ca129f01d149b5aab391353f90139b => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:49:48.416238 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8040b3a-e184-4891-9e6c-173e417ed3ee None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:48.540348 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ae8e0-7644-4f2b-871c-71474c654157 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:48.580716 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cb84cb-c8c0-49b1-a33c-53b8ae36f6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:48.581959 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb84cb-c8c0-49b1-a33c-53b8ae36f6d2 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:48.582452 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb84cb-c8c0-49b1-a33c-53b8ae36f6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:48.582931 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb84cb-c8c0-49b1-a33c-53b8ae36f6d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:48.583980 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cb84cb-c8c0-49b1-a33c-53b8ae36f6d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:48.585675 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cb84cb-c8c0-49b1-a33c-53b8ae36f6d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:48.587410 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cb84cb-c8c0-49b1-a33c-53b8ae36f6d2 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:48.599473 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 204/411] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:49:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:49:48.614083 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cb84cb-c8c0-49b1-a33c-53b8ae36f6d2 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:48.643391 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8040b3a-e184-4891-9e6c-173e417ed3ee None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:48.684595 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:48.717103 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9334df65-154e-4c20-8a9f-1cd7cf8f5c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:48.717993 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334df65-154e-4c20-8a9f-1cd7cf8f5c27 None admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:48.718279 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334df65-154e-4c20-8a9f-1cd7cf8f5c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:48.718558 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334df65-154e-4c20-8a9f-1cd7cf8f5c27 None admin] PATH_INFO: `/v3/projects/e8ca129f01d149b5aab391353f90139b/groups/e66391be7a2e4d2dab0bec9fcf9bdab7/roles/50e04e9d09e045b0bcf8e1782a6e2f23` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:48.723458 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9334df65-154e-4c20-8a9f-1cd7cf8f5c27 None admin] RBAC: Authorizing `identity:create_grant(project_id=e8ca129f01d149b5aab391353f90139b, group_id=e66391be7a2e4d2dab0bec9fcf9bdab7, role_id=50e04e9d09e045b0bcf8e1782a6e2f23)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:48.724965 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9334df65-154e-4c20-8a9f-1cd7cf8f5c27 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:48.756530 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 208/412] 10.222.0.13 () {58 vars in 1442 bytes} [Tue Jun 7 21:49:48 2022] PUT /identity/v3/projects/e8ca129f01d149b5aab391353f90139b/groups/e66391be7a2e4d2dab0bec9fcf9bdab7/roles/50e04e9d09e045b0bcf8e1782a6e2f23 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 21:49:48.798990 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cb84cb-c8c0-49b1-a33c-53b8ae36f6d2 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:48.853632 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b9fe6-da51-49e9-80f3-bffe85d6a98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:48.854502 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b9fe6-da51-49e9-80f3-bffe85d6a98c None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:48.854788 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b9fe6-da51-49e9-80f3-bffe85d6a98c None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:48.855065 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b9fe6-da51-49e9-80f3-bffe85d6a98c None admin] PATH_INFO: `/v3/groups/c01abaa0eb6d4bd5aa43790d53db8d32` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:48.856394 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106b9fe6-da51-49e9-80f3-bffe85d6a98c None admin] RBAC: Authorizing `identity:get_group(group_id=c01abaa0eb6d4bd5aa43790d53db8d32)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:48.857847 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106b9fe6-da51-49e9-80f3-bffe85d6a98c None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:48.860785 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 205/413] 10.222.0.13 () {56 vars in 1180 bytes} [Tue Jun 7 21:49:48 2022] GET /identity/v3/groups/c01abaa0eb6d4bd5aa43790d53db8d32 => generated 229 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:48.865612 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9334df65-154e-4c20-8a9f-1cd7cf8f5c27 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:48.886576 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:48.931388 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a475661a-c8c0-4c6b-8453-9987be239761 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:48.931729 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a475661a-c8c0-4c6b-8453-9987be239761 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:48.932018 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a475661a-c8c0-4c6b-8453-9987be239761 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:48.933455 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 206/414] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 21:49:48.944118 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f03c3-f919-48f3-b934-2dc23914cc8c None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:48.944428 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f03c3-f919-48f3-b934-2dc23914cc8c None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:48.944674 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f03c3-f919-48f3-b934-2dc23914cc8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:48.974171 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-d58f03c3-f919-48f3-b934-2dc23914cc8c None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:48.978391 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d58f03c3-f919-48f3-b934-2dc23914cc8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:49.049082 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9334df65-154e-4c20-8a9f-1cd7cf8f5c27 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:49.068385 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 207/415] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 21:49:49.106095 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58f03c3-f919-48f3-b934-2dc23914cc8c None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:49.133241 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1575ee-6981-4c1c-bf59-2a55cc56ce55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:49.134610 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1575ee-6981-4c1c-bf59-2a55cc56ce55 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:49.134905 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1575ee-6981-4c1c-bf59-2a55cc56ce55 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:49.135187 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1575ee-6981-4c1c-bf59-2a55cc56ce55 None admin] PATH_INFO: `/v3/projects/c61211fad6e447b0bf162f99f1111a82` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:49.136624 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1575ee-6981-4c1c-bf59-2a55cc56ce55 None admin] RBAC: Authorizing `identity:get_project(project_id=c61211fad6e447b0bf162f99f1111a82)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:49.138338 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1575ee-6981-4c1c-bf59-2a55cc56ce55 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:49.142649 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 209/416] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:49:48 2022] GET /identity/v3/projects/c61211fad6e447b0bf162f99f1111a82 => generated 344 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:49.150837 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1575ee-6981-4c1c-bf59-2a55cc56ce55 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:49.155142 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:49.252410 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:49.282201 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58f03c3-f919-48f3-b934-2dc23914cc8c None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:49.287518 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1575ee-6981-4c1c-bf59-2a55cc56ce55 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:49.334039 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d29ff9-fe4a-44bd-930f-7c095c4a8800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:49.334735 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d29ff9-fe4a-44bd-930f-7c095c4a8800 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:49.335030 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d29ff9-fe4a-44bd-930f-7c095c4a8800 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:49.335297 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d29ff9-fe4a-44bd-930f-7c095c4a8800 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:49.349307 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d29ff9-fe4a-44bd-930f-7c095c4a8800 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:49.357819 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-f5d29ff9-fe4a-44bd-930f-7c095c4a8800 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 21:49:49.357819 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:49.357819 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:49.357819 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:49.357819 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:49.357819 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:49.357819 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:49.357819 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:49.357819 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:49.357819 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:49.357819 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:49.357819 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:49.357819 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:49.357819 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:49.357819 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:49.357819 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:49.358921 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:49.358921 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:49.358921 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:49.358921 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:49.358921 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:49.358921 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:49.358921 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:49.358921 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:49.358921 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:49.358921 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:49.358921 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:49.358921 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:49.358921 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:49.358921 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:49.358921 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:49.358921 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:49.359936 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:49.359936 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:49.359936 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:49.359936 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:49.359936 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:49.359936 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:49.359936 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:49.359936 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:49.359936 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:49.359936 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:49.359936 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 21:49:49.359936 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:49:49.363807 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 208/417] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:49:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:49:49.366494 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d29ff9-fe4a-44bd-930f-7c095c4a8800 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:49.375727 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb5c45-3a06-47c3-a926-fe9ed10f5351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:49.376891 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb5c45-3a06-47c3-a926-fe9ed10f5351 None admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:49.377321 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb5c45-3a06-47c3-a926-fe9ed10f5351 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:49.377744 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb5c45-3a06-47c3-a926-fe9ed10f5351 None admin] PATH_INFO: `/v3/projects/c61211fad6e447b0bf162f99f1111a82/groups/c01abaa0eb6d4bd5aa43790d53db8d32/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:49.384149 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bb5c45-3a06-47c3-a926-fe9ed10f5351 None admin] RBAC: Authorizing `identity:create_grant(project_id=c61211fad6e447b0bf162f99f1111a82, group_id=c01abaa0eb6d4bd5aa43790d53db8d32, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:49.385853 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bb5c45-3a06-47c3-a926-fe9ed10f5351 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:49.418950 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 210/418] 10.222.0.13 () {58 vars in 1442 bytes} [Tue Jun 7 21:49:49 2022] PUT /identity/v3/projects/c61211fad6e447b0bf162f99f1111a82/groups/c01abaa0eb6d4bd5aa43790d53db8d32/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 21:49:49.507239 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:49.515533 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d29ff9-fe4a-44bd-930f-7c095c4a8800 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:49.560767 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ef27e-9b92-4656-a40d-0f586173fcec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:49.561480 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ef27e-9b92-4656-a40d-0f586173fcec None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:49.561768 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ef27e-9b92-4656-a40d-0f586173fcec None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:49.562044 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ef27e-9b92-4656-a40d-0f586173fcec None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:49.562807 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ef27e-9b92-4656-a40d-0f586173fcec None admin] RBAC: Adding query filter params (name=member) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:49.563119 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ef27e-9b92-4656-a40d-0f586173fcec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:49.564598 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ef27e-9b92-4656-a40d-0f586173fcec None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:49.574268 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 209/419] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jun 7 21:49:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:49:49.579636 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb5c45-3a06-47c3-a926-fe9ed10f5351 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:49.712771 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb5c45-3a06-47c3-a926-fe9ed10f5351 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:49.748464 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e53170-65e1-4c50-a672-ec0bff752858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:49.749451 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e53170-65e1-4c50-a672-ec0bff752858 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:49.749906 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e53170-65e1-4c50-a672-ec0bff752858 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:49.750349 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e53170-65e1-4c50-a672-ec0bff752858 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:49.776449 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:49.847568 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e53170-65e1-4c50-a672-ec0bff752858 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:49.849720 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e53170-65e1-4c50-a672-ec0bff752858 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:49.896592 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f0d628-bb55-4dfe-9233-4d6504e49ea1 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:49.897121 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f0d628-bb55-4dfe-9233-4d6504e49ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:49.897121 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f0d628-bb55-4dfe-9233-4d6504e49ea1 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:49.898733 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 210/420] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:49.903861 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734c19e1-ae8e-4422-9546-a6c6b9ca7bcd None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:49.904150 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734c19e1-ae8e-4422-9546-a6c6b9ca7bcd None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:49.904431 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734c19e1-ae8e-4422-9546-a6c6b9ca7bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:49.923788 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-734c19e1-ae8e-4422-9546-a6c6b9ca7bcd None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:49.925133 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-734c19e1-ae8e-4422-9546-a6c6b9ca7bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:50.022966 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 211/421] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:49:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 21:49:50.029330 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e53170-65e1-4c50-a672-ec0bff752858 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:50.078030 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 211/422] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 21:49:50.095776 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:50.124454 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734c19e1-ae8e-4422-9546-a6c6b9ca7bcd None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:50.177542 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e53170-65e1-4c50-a672-ec0bff752858 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:50.234009 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:50.244152 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6550436-2107-45ba-a2bb-08ec417e6ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:50.245030 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6550436-2107-45ba-a2bb-08ec417e6ffb None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:50.245311 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6550436-2107-45ba-a2bb-08ec417e6ffb None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:50.245580 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6550436-2107-45ba-a2bb-08ec417e6ffb None admin] PATH_INFO: `/v3/projects/b1e275c904b64687b29e7ca3d28fdda7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:50.247381 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6550436-2107-45ba-a2bb-08ec417e6ffb None admin] RBAC: Authorizing `identity:get_project(project_id=b1e275c904b64687b29e7ca3d28fdda7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:50.248824 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6550436-2107-45ba-a2bb-08ec417e6ffb None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:50.256927 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 212/423] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:49:50 2022] GET /identity/v3/projects/b1e275c904b64687b29e7ca3d28fdda7 => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:49:50.258825 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6550436-2107-45ba-a2bb-08ec417e6ffb None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:50.266221 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734c19e1-ae8e-4422-9546-a6c6b9ca7bcd None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:50.324756 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03f861-4284-4bcc-80a6-3cc08dedc73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:50.325925 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03f861-4284-4bcc-80a6-3cc08dedc73f None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:50.326551 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03f861-4284-4bcc-80a6-3cc08dedc73f None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:50.326997 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03f861-4284-4bcc-80a6-3cc08dedc73f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:50.340104 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f03f861-4284-4bcc-80a6-3cc08dedc73f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:50.350775 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-4f03f861-4284-4bcc-80a6-3cc08dedc73f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 21:49:50.350775 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:50.350775 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:50.350775 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:50.350775 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:50.350775 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:50.350775 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:50.350775 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:50.350775 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:50.350775 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:50.350775 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:50.350775 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:50.350775 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:50.350775 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:50.350775 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:50.350775 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:50.351710 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:50.351710 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:50.351710 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:50.351710 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:50.351710 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:50.351710 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:50.351710 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:50.351710 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:50.351710 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:50.351710 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:50.351710 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:50.351710 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:50.351710 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:50.351710 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:50.351710 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:50.351710 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:50.352600 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:50.352600 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:50.352600 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:50.352600 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:50.352600 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:50.352600 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:50.352600 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:50.352600 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:50.352600 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:50.352600 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:50.352600 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 21:49:50.352600 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:49:50.355004 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 212/424] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Jun 7 21:49:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:49:50.362418 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03f861-4284-4bcc-80a6-3cc08dedc73f None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:50.434399 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6550436-2107-45ba-a2bb-08ec417e6ffb None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:50.451967 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:50.506430 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a619da-7bfe-4898-b76e-2f2fffda0947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:50.507301 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a619da-7bfe-4898-b76e-2f2fffda0947 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:50.507608 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a619da-7bfe-4898-b76e-2f2fffda0947 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:50.507888 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a619da-7bfe-4898-b76e-2f2fffda0947 None admin] PATH_INFO: `/v3/groups/e66391be7a2e4d2dab0bec9fcf9bdab7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:50.510191 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a619da-7bfe-4898-b76e-2f2fffda0947 None admin] RBAC: Authorizing `identity:get_group(group_id=e66391be7a2e4d2dab0bec9fcf9bdab7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:50.511684 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a619da-7bfe-4898-b76e-2f2fffda0947 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:50.514252 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 213/425] 10.222.0.13 () {56 vars in 1180 bytes} [Tue Jun 7 21:49:50 2022] GET /identity/v3/groups/e66391be7a2e4d2dab0bec9fcf9bdab7 => generated 226 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:50.515726 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5de3d-9963-49b9-8a56-f53907efdccc None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:50.516015 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5de3d-9963-49b9-8a56-f53907efdccc None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:50.516290 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5de3d-9963-49b9-8a56-f53907efdccc None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:50.517573 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 214/426] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:50.521127 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f5de3d-9963-49b9-8a56-f53907efdccc None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:50.561815 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03f861-4284-4bcc-80a6-3cc08dedc73f None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:50.598878 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384dede5-470b-49f2-b2d4-51e97150ba1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:50.599533 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384dede5-470b-49f2-b2d4-51e97150ba1d None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:50.599826 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384dede5-470b-49f2-b2d4-51e97150ba1d None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:50.600105 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384dede5-470b-49f2-b2d4-51e97150ba1d None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:50.600924 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384dede5-470b-49f2-b2d4-51e97150ba1d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:50.601276 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384dede5-470b-49f2-b2d4-51e97150ba1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:50.602662 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384dede5-470b-49f2-b2d4-51e97150ba1d None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:50.615605 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 213/427] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:49:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:50.616981 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8dfc51-1bcd-463e-a55b-8101fd7ee021 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:50.617270 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8dfc51-1bcd-463e-a55b-8101fd7ee021 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:50.617552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8dfc51-1bcd-463e-a55b-8101fd7ee021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:50.636058 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-8b8dfc51-1bcd-463e-a55b-8101fd7ee021 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:50.637255 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-8b8dfc51-1bcd-463e-a55b-8101fd7ee021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:50.639475 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:50.656057 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f5de3d-9963-49b9-8a56-f53907efdccc None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:50.698470 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572a2f47-3079-4646-b43f-1433dcf7fd4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:50.699174 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a2f47-3079-4646-b43f-1433dcf7fd4a None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:50.699523 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a2f47-3079-4646-b43f-1433dcf7fd4a None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:50.699803 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a2f47-3079-4646-b43f-1433dcf7fd4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:50.700630 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572a2f47-3079-4646-b43f-1433dcf7fd4a None admin] RBAC: Adding query filter params (group.id=e66391be7a2e4d2dab0bec9fcf9bdab7, role.id=c5431c9deadf47e496264ba5b78eeaf6, scope.project.id=b1e275c904b64687b29e7ca3d28fdda7) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:50.700994 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572a2f47-3079-4646-b43f-1433dcf7fd4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:50.702405 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572a2f47-3079-4646-b43f-1433dcf7fd4a None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:50.709214 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 215/428] 10.222.0.13 () {56 vars in 1376 bytes} [Tue Jun 7 21:49:50 2022] GET /identity/v3/role_assignments?group.id=e66391be7a2e4d2dab0bec9fcf9bdab7&scope.project.id=b1e275c904b64687b29e7ca3d28fdda7&role.id=c5431c9deadf47e496264ba5b78eeaf6 => generated 611 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:49:50.710033 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572a2f47-3079-4646-b43f-1433dcf7fd4a None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:50.725921 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 214/429] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:49:50.795014 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8dfc51-1bcd-463e-a55b-8101fd7ee021 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:50.795254 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:50.854724 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572a2f47-3079-4646-b43f-1433dcf7fd4a None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:50.892502 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc6e263-a034-4c7c-910b-c9ff79a7f784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:50.893228 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6e263-a034-4c7c-910b-c9ff79a7f784 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:50.893510 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6e263-a034-4c7c-910b-c9ff79a7f784 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:50.894352 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6e263-a034-4c7c-910b-c9ff79a7f784 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:50.971319 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc6e263-a034-4c7c-910b-c9ff79a7f784 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:50.973809 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc6e263-a034-4c7c-910b-c9ff79a7f784 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:51.023292 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8dfc51-1bcd-463e-a55b-8101fd7ee021 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:51.027719 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:51.076331 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1e41de-3263-4b4a-9a28-d463084ffc23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:51.077523 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e41de-3263-4b4a-9a28-d463084ffc23 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:51.077944 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e41de-3263-4b4a-9a28-d463084ffc23 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:51.078355 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e41de-3263-4b4a-9a28-d463084ffc23 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:51.086105 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 216/430] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:49:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 21:49:51.091778 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc6e263-a034-4c7c-910b-c9ff79a7f784 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:51.092467 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:51.098555 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1e41de-3263-4b4a-9a28-d463084ffc23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:51.110344 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-1f1e41de-3263-4b4a-9a28-d463084ffc23 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 21:49:51.110344 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:51.110344 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:51.110344 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:51.110344 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:51.110344 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:51.110344 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:51.110344 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:51.110344 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:51.110344 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:51.110344 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:51.110344 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:51.110344 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:51.110344 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:51.110344 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:51.110344 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:51.111260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:51.111260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:51.111260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:51.111260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:51.111260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:51.111260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:51.111260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:51.111260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:51.111260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:51.111260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:51.111260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:51.111260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:51.111260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:51.111260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:51.111260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:51.111260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:51.112092 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:51.112092 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:51.112092 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:51.112092 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:51.112092 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:51.112092 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:51.112092 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:51.112092 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:51.112092 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:51.112092 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:51.112092 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 21:49:51.112092 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:49:51.116526 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 215/431] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:49:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:49:51.122249 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1e41de-3263-4b4a-9a28-d463084ffc23 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:51.209815 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc6e263-a034-4c7c-910b-c9ff79a7f784 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:51.234106 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1e41de-3263-4b4a-9a28-d463084ffc23 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:51.251699 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b63a18-b907-48ed-ba61-e24d2b91465a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:51.253602 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b63a18-b907-48ed-ba61-e24d2b91465a None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:51.254257 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b63a18-b907-48ed-ba61-e24d2b91465a None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:51.254751 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b63a18-b907-48ed-ba61-e24d2b91465a None admin] PATH_INFO: `/v3/projects/b1e275c904b64687b29e7ca3d28fdda7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:51.256219 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b63a18-b907-48ed-ba61-e24d2b91465a None admin] RBAC: Authorizing `identity:get_project(project_id=b1e275c904b64687b29e7ca3d28fdda7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:51.258069 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b63a18-b907-48ed-ba61-e24d2b91465a None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:51.260120 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 217/432] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:49:51 2022] GET /identity/v3/projects/b1e275c904b64687b29e7ca3d28fdda7 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:49:51.265807 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b63a18-b907-48ed-ba61-e24d2b91465a None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:51.272760 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce02c590-e0d4-4dd7-ac73-7f2d1626d35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:51.273432 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02c590-e0d4-4dd7-ac73-7f2d1626d35e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:51.273711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02c590-e0d4-4dd7-ac73-7f2d1626d35e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:51.273983 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02c590-e0d4-4dd7-ac73-7f2d1626d35e None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:51.274734 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce02c590-e0d4-4dd7-ac73-7f2d1626d35e None admin] RBAC: Adding query filter params (name=member) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:51.275038 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce02c590-e0d4-4dd7-ac73-7f2d1626d35e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:51.276440 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce02c590-e0d4-4dd7-ac73-7f2d1626d35e None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:51.290816 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 216/433] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jun 7 21:49:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:51.292273 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce02c590-e0d4-4dd7-ac73-7f2d1626d35e None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:51.299720 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:51.359544 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:51.397014 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b63a18-b907-48ed-ba61-e24d2b91465a None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:51.414770 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce02c590-e0d4-4dd7-ac73-7f2d1626d35e None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:51.439377 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab7b35-bc7c-4f49-b135-e10762304408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:51.441257 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab7b35-bc7c-4f49-b135-e10762304408 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:51.442012 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab7b35-bc7c-4f49-b135-e10762304408 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:51.442560 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab7b35-bc7c-4f49-b135-e10762304408 None admin] PATH_INFO: `/v3/groups/e66391be7a2e4d2dab0bec9fcf9bdab7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:51.444123 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ab7b35-bc7c-4f49-b135-e10762304408 None admin] RBAC: Authorizing `identity:get_group(group_id=e66391be7a2e4d2dab0bec9fcf9bdab7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:51.445866 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ab7b35-bc7c-4f49-b135-e10762304408 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:51.447856 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 218/434] 10.222.0.13 () {56 vars in 1180 bytes} [Tue Jun 7 21:49:51 2022] GET /identity/v3/groups/e66391be7a2e4d2dab0bec9fcf9bdab7 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:49:51.454064 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab7b35-bc7c-4f49-b135-e10762304408 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:51.454579 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e51253-a343-425a-8121-d1c544a32ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:51.455166 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e51253-a343-425a-8121-d1c544a32ad5 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:51.455469 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e51253-a343-425a-8121-d1c544a32ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:51.455745 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e51253-a343-425a-8121-d1c544a32ad5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:51.526951 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e51253-a343-425a-8121-d1c544a32ad5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:51.529367 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e51253-a343-425a-8121-d1c544a32ad5 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:51.551756 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:51.572307 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab7b35-bc7c-4f49-b135-e10762304408 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:51.591293 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:51.616357 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859aa9a5-9ef6-43e4-bd42-45eb5700eeb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:51.617528 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859aa9a5-9ef6-43e4-bd42-45eb5700eeb6 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:51.618105 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859aa9a5-9ef6-43e4-bd42-45eb5700eeb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:51.618613 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859aa9a5-9ef6-43e4-bd42-45eb5700eeb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:51.619704 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859aa9a5-9ef6-43e4-bd42-45eb5700eeb6 None admin] RBAC: Adding query filter params (group.id=e66391be7a2e4d2dab0bec9fcf9bdab7, role.id=50e04e9d09e045b0bcf8e1782a6e2f23, scope.project.id=b1e275c904b64687b29e7ca3d28fdda7) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:51.620241 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859aa9a5-9ef6-43e4-bd42-45eb5700eeb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:51.622054 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859aa9a5-9ef6-43e4-bd42-45eb5700eeb6 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:51.629497 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 219/435] 10.222.0.13 () {56 vars in 1376 bytes} [Tue Jun 7 21:49:51 2022] GET /identity/v3/role_assignments?group.id=e66391be7a2e4d2dab0bec9fcf9bdab7&scope.project.id=b1e275c904b64687b29e7ca3d28fdda7&role.id=50e04e9d09e045b0bcf8e1782a6e2f23 => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:51.642076 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 217/436] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:49:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 21:49:51.644180 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859aa9a5-9ef6-43e4-bd42-45eb5700eeb6 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:51.770644 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859aa9a5-9ef6-43e4-bd42-45eb5700eeb6 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:51.797017 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfef3a2-da15-4131-acd3-adac1de45c0c None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:51.797307 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfef3a2-da15-4131-acd3-adac1de45c0c None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:51.797588 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfef3a2-da15-4131-acd3-adac1de45c0c None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:51.799319 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 218/437] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:51.805824 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd565d-c2f5-4d5e-a1d5-66032e0af244 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:51.806112 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd565d-c2f5-4d5e-a1d5-66032e0af244 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:51.806390 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd565d-c2f5-4d5e-a1d5-66032e0af244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:51.814670 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223e181-615f-40a4-a7a6-8e2ec246d4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:51.816293 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223e181-615f-40a4-a7a6-8e2ec246d4fa None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:51.816677 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223e181-615f-40a4-a7a6-8e2ec246d4fa None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:51.817050 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223e181-615f-40a4-a7a6-8e2ec246d4fa None admin] PATH_INFO: `/v3/projects/e8ca129f01d149b5aab391353f90139b` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:51.818548 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8223e181-615f-40a4-a7a6-8e2ec246d4fa None admin] RBAC: Authorizing `identity:get_project(project_id=e8ca129f01d149b5aab391353f90139b)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:51.820404 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8223e181-615f-40a4-a7a6-8e2ec246d4fa None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:51.822579 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 220/438] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:49:51 2022] GET /identity/v3/projects/e8ca129f01d149b5aab391353f90139b => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:51.825259 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-defd565d-c2f5-4d5e-a1d5-66032e0af244 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:51.826535 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-defd565d-c2f5-4d5e-a1d5-66032e0af244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:51.828773 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223e181-615f-40a4-a7a6-8e2ec246d4fa None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:51.852263 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:51.916862 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 219/439] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 21:49:51.941876 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd565d-c2f5-4d5e-a1d5-66032e0af244 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:51.972890 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223e181-615f-40a4-a7a6-8e2ec246d4fa None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:52.011073 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7cffa7-066e-43a8-bf97-e777e3dd2acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:52.011971 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7cffa7-066e-43a8-bf97-e777e3dd2acf None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:52.012255 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7cffa7-066e-43a8-bf97-e777e3dd2acf None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:52.012533 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7cffa7-066e-43a8-bf97-e777e3dd2acf None admin] PATH_INFO: `/v3/groups/e66391be7a2e4d2dab0bec9fcf9bdab7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:52.013733 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7cffa7-066e-43a8-bf97-e777e3dd2acf None admin] RBAC: Authorizing `identity:get_group(group_id=e66391be7a2e4d2dab0bec9fcf9bdab7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:52.015235 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7cffa7-066e-43a8-bf97-e777e3dd2acf None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:52.016852 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 221/440] 10.222.0.13 () {56 vars in 1180 bytes} [Tue Jun 7 21:49:51 2022] GET /identity/v3/groups/e66391be7a2e4d2dab0bec9fcf9bdab7 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:52.023083 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7cffa7-066e-43a8-bf97-e777e3dd2acf None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:52.056556 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:52.072546 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd565d-c2f5-4d5e-a1d5-66032e0af244 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:52.114760 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff324326-53f2-4b66-8ebb-6483bac5e476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:52.115615 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff324326-53f2-4b66-8ebb-6483bac5e476 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:52.115905 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff324326-53f2-4b66-8ebb-6483bac5e476 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:52.116185 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff324326-53f2-4b66-8ebb-6483bac5e476 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:52.124370 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:52.132894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff324326-53f2-4b66-8ebb-6483bac5e476 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:52.142471 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-ff324326-53f2-4b66-8ebb-6483bac5e476 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 21:49:52.142471 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:52.142471 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:52.142471 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:52.142471 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:52.142471 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:52.142471 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:52.142471 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:52.142471 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:52.142471 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:52.142471 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:52.142471 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:52.142471 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:52.142471 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:52.142471 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:52.142471 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:52.143608 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:52.143608 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:52.143608 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:52.143608 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:52.143608 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:52.143608 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:52.143608 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:52.143608 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:52.143608 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:52.143608 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:52.143608 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:52.143608 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:52.143608 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:52.143608 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:52.143608 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:52.143608 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:52.144520 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:52.144520 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:52.144520 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:52.144520 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:52.144520 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:52.144520 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:52.144520 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:52.144520 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:52.144520 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:52.144520 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:52.144520 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 21:49:52.144520 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:49:52.144520 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 220/441] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Jun 7 21:49:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:49:52.151457 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff324326-53f2-4b66-8ebb-6483bac5e476 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:52.153262 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7cffa7-066e-43a8-bf97-e777e3dd2acf None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:52.192449 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f59f5-7e3c-4b72-9445-5dbf8f331877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:52.193260 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f59f5-7e3c-4b72-9445-5dbf8f331877 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:52.194301 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f59f5-7e3c-4b72-9445-5dbf8f331877 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:52.194301 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f59f5-7e3c-4b72-9445-5dbf8f331877 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:52.194859 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3f59f5-7e3c-4b72-9445-5dbf8f331877 None admin] RBAC: Adding query filter params (group.id=e66391be7a2e4d2dab0bec9fcf9bdab7, role.id=c5431c9deadf47e496264ba5b78eeaf6, scope.project.id=e8ca129f01d149b5aab391353f90139b) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:52.195234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3f59f5-7e3c-4b72-9445-5dbf8f331877 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:52.196748 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3f59f5-7e3c-4b72-9445-5dbf8f331877 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:52.203080 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 222/442] 10.222.0.13 () {56 vars in 1376 bytes} [Tue Jun 7 21:49:52 2022] GET /identity/v3/role_assignments?group.id=e66391be7a2e4d2dab0bec9fcf9bdab7&scope.project.id=e8ca129f01d149b5aab391353f90139b&role.id=c5431c9deadf47e496264ba5b78eeaf6 => generated 611 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:49:52.231412 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea04931-8f48-4a33-b870-e23085f693f3 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:52.231941 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea04931-8f48-4a33-b870-e23085f693f3 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:52.232222 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea04931-8f48-4a33-b870-e23085f693f3 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:52.233851 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 223/443] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 21:49:52.240408 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6d36f-e242-48a3-9643-e73555158315 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:52.240702 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6d36f-e242-48a3-9643-e73555158315 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:52.240978 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6d36f-e242-48a3-9643-e73555158315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:52.258729 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-70b6d36f-e242-48a3-9643-e73555158315 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:52.259965 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-70b6d36f-e242-48a3-9643-e73555158315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:52.272772 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff324326-53f2-4b66-8ebb-6483bac5e476 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:52.314164 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6327073-162c-4afe-83bc-0816cae836cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:52.314845 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6327073-162c-4afe-83bc-0816cae836cc None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:52.315127 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6327073-162c-4afe-83bc-0816cae836cc None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:52.315456 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6327073-162c-4afe-83bc-0816cae836cc None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:52.316232 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6327073-162c-4afe-83bc-0816cae836cc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:52.316540 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6327073-162c-4afe-83bc-0816cae836cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:52.318085 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6327073-162c-4afe-83bc-0816cae836cc None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:52.326915 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 221/444] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:49:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:49:52.332504 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6327073-162c-4afe-83bc-0816cae836cc None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:52.339839 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 224/445] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:49:52.351358 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:52.365103 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6d36f-e242-48a3-9643-e73555158315 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:52.449285 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6327073-162c-4afe-83bc-0816cae836cc None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:52.468149 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:52.495483 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6d36f-e242-48a3-9643-e73555158315 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:52.497448 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96fa5f-c167-4602-ab42-56fca34aadf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:52.498889 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96fa5f-c167-4602-ab42-56fca34aadf6 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:52.499180 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96fa5f-c167-4602-ab42-56fca34aadf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:52.499489 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96fa5f-c167-4602-ab42-56fca34aadf6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:52.537735 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d447b5-0918-43cb-8ba7-2c144351c92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:52.539034 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d447b5-0918-43cb-8ba7-2c144351c92a None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:52.539272 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d447b5-0918-43cb-8ba7-2c144351c92a None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:52.540756 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d447b5-0918-43cb-8ba7-2c144351c92a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:52.551801 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d447b5-0918-43cb-8ba7-2c144351c92a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:52.559732 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-80d447b5-0918-43cb-8ba7-2c144351c92a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:49:52.559732 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:52.559732 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:52.559732 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:52.559732 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:52.559732 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:52.559732 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:52.559732 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:52.559732 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:52.559732 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:52.559732 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:52.559732 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:52.559732 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:52.559732 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:52.559732 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:52.559732 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:52.560813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:52.560813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:52.560813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:52.560813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:52.560813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:52.560813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:52.560813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:52.560813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:52.560813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:52.560813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:52.560813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:52.560813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:52.560813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:52.560813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:52.560813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:52.560813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:52.561813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:52.561813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:52.561813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:52.561813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:52.561813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:52.561813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:52.561813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:52.561813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:52.561813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:52.561813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:52.561813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:49:52.561813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:49:52.561813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 225/446] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jun 7 21:49:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:49:52.567118 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d447b5-0918-43cb-8ba7-2c144351c92a None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:52.586512 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea96fa5f-c167-4602-ab42-56fca34aadf6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:52.588049 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea96fa5f-c167-4602-ab42-56fca34aadf6 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:52.608499 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:52.675582 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d447b5-0918-43cb-8ba7-2c144351c92a None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:52.698732 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 222/447] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:49:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 21:49:52.704267 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96fa5f-c167-4602-ab42-56fca34aadf6 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:52.709617 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682802f-e97b-40c9-a60d-2ca261bcdf75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:52.710284 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682802f-e97b-40c9-a60d-2ca261bcdf75 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:52.710567 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682802f-e97b-40c9-a60d-2ca261bcdf75 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:52.710845 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682802f-e97b-40c9-a60d-2ca261bcdf75 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:52.711615 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b682802f-e97b-40c9-a60d-2ca261bcdf75 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:52.711920 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b682802f-e97b-40c9-a60d-2ca261bcdf75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:52.713293 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b682802f-e97b-40c9-a60d-2ca261bcdf75 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:52.715452 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:52.722303 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 226/448] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:49:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:49:52.727755 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682802f-e97b-40c9-a60d-2ca261bcdf75 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:52.802255 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96fa5f-c167-4602-ab42-56fca34aadf6 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:52.815045 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:52.828248 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682802f-e97b-40c9-a60d-2ca261bcdf75 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:52.838208 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7128f83d-0c88-4dbc-99e0-af75c861e5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:52.839073 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128f83d-0c88-4dbc-99e0-af75c861e5ab None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:52.839382 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128f83d-0c88-4dbc-99e0-af75c861e5ab None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:52.839654 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128f83d-0c88-4dbc-99e0-af75c861e5ab None admin] PATH_INFO: `/v3/projects/e8ca129f01d149b5aab391353f90139b` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:52.840848 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7128f83d-0c88-4dbc-99e0-af75c861e5ab None admin] RBAC: Authorizing `identity:get_project(project_id=e8ca129f01d149b5aab391353f90139b)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:52.842298 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7128f83d-0c88-4dbc-99e0-af75c861e5ab None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:52.844015 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 223/449] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:49:52 2022] GET /identity/v3/projects/e8ca129f01d149b5aab391353f90139b => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:49:52.849735 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7128f83d-0c88-4dbc-99e0-af75c861e5ab None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:52.861680 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b861ba-4de9-4c7b-a402-658b291d26fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:52.862359 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b861ba-4de9-4c7b-a402-658b291d26fb None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:52.862636 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b861ba-4de9-4c7b-a402-658b291d26fb None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:52.862915 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b861ba-4de9-4c7b-a402-658b291d26fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:52.926203 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:52.930129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b861ba-4de9-4c7b-a402-658b291d26fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:52.931639 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b861ba-4de9-4c7b-a402-658b291d26fb None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:52.947472 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7128f83d-0c88-4dbc-99e0-af75c861e5ab None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:52.980703 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9564525-129d-4e60-91f4-2813bdfc22d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:52.981545 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9564525-129d-4e60-91f4-2813bdfc22d6 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:52.981825 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9564525-129d-4e60-91f4-2813bdfc22d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:52.982096 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9564525-129d-4e60-91f4-2813bdfc22d6 None admin] PATH_INFO: `/v3/groups/e66391be7a2e4d2dab0bec9fcf9bdab7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:52.983259 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9564525-129d-4e60-91f4-2813bdfc22d6 None admin] RBAC: Authorizing `identity:get_group(group_id=e66391be7a2e4d2dab0bec9fcf9bdab7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:52.984711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9564525-129d-4e60-91f4-2813bdfc22d6 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:52.986299 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 224/450] 10.222.0.13 () {56 vars in 1180 bytes} [Tue Jun 7 21:49:52 2022] GET /identity/v3/groups/e66391be7a2e4d2dab0bec9fcf9bdab7 => generated 226 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:49:52.992082 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9564525-129d-4e60-91f4-2813bdfc22d6 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:53.030789 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 227/451] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:49:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 21:49:53.036391 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b861ba-4de9-4c7b-a402-658b291d26fb None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:53.040002 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:53.090087 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9564525-129d-4e60-91f4-2813bdfc22d6 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:53.123377 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d2ad62-5368-47ed-829e-3fe951f1fc07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:53.124017 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2ad62-5368-47ed-829e-3fe951f1fc07 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:53.124298 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2ad62-5368-47ed-829e-3fe951f1fc07 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:53.124568 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2ad62-5368-47ed-829e-3fe951f1fc07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:53.125384 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d2ad62-5368-47ed-829e-3fe951f1fc07 None admin] RBAC: Adding query filter params (group.id=e66391be7a2e4d2dab0bec9fcf9bdab7, role.id=50e04e9d09e045b0bcf8e1782a6e2f23, scope.project.id=e8ca129f01d149b5aab391353f90139b) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:53.125689 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d2ad62-5368-47ed-829e-3fe951f1fc07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:53.127207 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d2ad62-5368-47ed-829e-3fe951f1fc07 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:53.130145 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:53.133784 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 225/452] 10.222.0.13 () {56 vars in 1376 bytes} [Tue Jun 7 21:49:52 2022] GET /identity/v3/role_assignments?group.id=e66391be7a2e4d2dab0bec9fcf9bdab7&scope.project.id=e8ca129f01d149b5aab391353f90139b&role.id=50e04e9d09e045b0bcf8e1782a6e2f23 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:49:53.133940 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b861ba-4de9-4c7b-a402-658b291d26fb None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:53.171001 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3672317b-34f4-4eb4-a07d-d297b05da396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:53.171908 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672317b-34f4-4eb4-a07d-d297b05da396 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:53.172190 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672317b-34f4-4eb4-a07d-d297b05da396 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:53.172465 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672317b-34f4-4eb4-a07d-d297b05da396 None admin] PATH_INFO: `/v3/projects/c61211fad6e447b0bf162f99f1111a82` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:53.173714 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3672317b-34f4-4eb4-a07d-d297b05da396 None admin] RBAC: Authorizing `identity:get_project(project_id=c61211fad6e447b0bf162f99f1111a82)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:53.175223 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3672317b-34f4-4eb4-a07d-d297b05da396 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:53.177242 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 228/453] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:49:53 2022] GET /identity/v3/projects/c61211fad6e447b0bf162f99f1111a82 => generated 344 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:49:53.182970 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d2ad62-5368-47ed-829e-3fe951f1fc07 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:53.287980 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d2ad62-5368-47ed-829e-3fe951f1fc07 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:53.301358 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:53.323001 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937bed1b-9560-4384-8017-6341165d24fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:53.323887 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937bed1b-9560-4384-8017-6341165d24fe None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:53.324170 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937bed1b-9560-4384-8017-6341165d24fe None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:53.324448 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937bed1b-9560-4384-8017-6341165d24fe None admin] PATH_INFO: `/v3/groups/c01abaa0eb6d4bd5aa43790d53db8d32` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:53.325629 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937bed1b-9560-4384-8017-6341165d24fe None admin] RBAC: Authorizing `identity:get_group(group_id=c01abaa0eb6d4bd5aa43790d53db8d32)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:53.327080 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937bed1b-9560-4384-8017-6341165d24fe None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:53.328859 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 226/454] 10.222.0.13 () {56 vars in 1180 bytes} [Tue Jun 7 21:49:53 2022] GET /identity/v3/groups/c01abaa0eb6d4bd5aa43790d53db8d32 => generated 229 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:53.335196 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3672317b-34f4-4eb4-a07d-d297b05da396 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:53.439378 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3672317b-34f4-4eb4-a07d-d297b05da396 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:53.476042 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a30ca-247f-4707-8d6f-551fa34c2c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:53.476705 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a30ca-247f-4707-8d6f-551fa34c2c04 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:53.476988 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a30ca-247f-4707-8d6f-551fa34c2c04 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:53.477265 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a30ca-247f-4707-8d6f-551fa34c2c04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:53.478106 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a30ca-247f-4707-8d6f-551fa34c2c04 None admin] RBAC: Adding query filter params (group.id=c01abaa0eb6d4bd5aa43790d53db8d32, role.id=6b483758dbd44eb4a78df2ce4603d601, scope.project.id=c61211fad6e447b0bf162f99f1111a82) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:53.478421 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a30ca-247f-4707-8d6f-551fa34c2c04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:53.479908 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a30ca-247f-4707-8d6f-551fa34c2c04 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:53.485920 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 229/455] 10.222.0.13 () {56 vars in 1376 bytes} [Tue Jun 7 21:49:53 2022] GET /identity/v3/role_assignments?group.id=c01abaa0eb6d4bd5aa43790d53db8d32&scope.project.id=c61211fad6e447b0bf162f99f1111a82&role.id=6b483758dbd44eb4a78df2ce4603d601 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:56.189109 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ed3f6-27d6-4e28-831b-cb236850845e None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:56.190273 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ed3f6-27d6-4e28-831b-cb236850845e None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:56.191362 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ed3f6-27d6-4e28-831b-cb236850845e None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:56.193405 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 227/456] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:56.199788 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee3cf8-98c5-4a53-913e-97d40ab7aa7a None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:56.200833 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee3cf8-98c5-4a53-913e-97d40ab7aa7a None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:56.201346 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee3cf8-98c5-4a53-913e-97d40ab7aa7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:56.228007 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-5bee3cf8-98c5-4a53-913e-97d40ab7aa7a None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:56.229618 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-5bee3cf8-98c5-4a53-913e-97d40ab7aa7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:56.266751 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:56.324710 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 230/457] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:49:56.364796 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ed3f6-27d6-4e28-831b-cb236850845e None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:56.489249 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ed3f6-27d6-4e28-831b-cb236850845e None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:56.541690 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c09eb2-d7ec-458e-9468-c86bd1e71ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:56.542861 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c09eb2-d7ec-458e-9468-c86bd1e71ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:56.543375 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c09eb2-d7ec-458e-9468-c86bd1e71ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:56.543871 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c09eb2-d7ec-458e-9468-c86bd1e71ce5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:56.555719 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c09eb2-d7ec-458e-9468-c86bd1e71ce5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:56.557492 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c09eb2-d7ec-458e-9468-c86bd1e71ce5 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:56.578960 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-b2c09eb2-d7ec-458e-9468-c86bd1e71ce5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:49:56.578960 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:56.578960 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:56.578960 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:56.578960 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:56.578960 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:56.578960 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:56.578960 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:56.578960 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:56.578960 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:56.578960 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:56.578960 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:56.578960 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:49:56.578960 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:49:56.578960 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:49:56.579936 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:49:56.579936 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:56.579936 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:56.579936 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:56.579936 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:56.579936 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:56.579936 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:56.579936 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:56.579936 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:56.579936 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:56.579936 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:56.579936 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:56.579936 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:56.579936 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:56.579936 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:56.579936 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:56.579936 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:56.580780 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:49:56.580780 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:49:56.580780 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:49:56.580780 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:49:56.583409 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 228/458] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:49:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:49:56.591030 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee3cf8-98c5-4a53-913e-97d40ab7aa7a None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:56.614357 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439dad5-ff28-4055-bdd5-08664c1bb0a5 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:56.614659 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439dad5-ff28-4055-bdd5-08664c1bb0a5 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:56.614909 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439dad5-ff28-4055-bdd5-08664c1bb0a5 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:56.616658 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 229/459] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 21:49:56.617727 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee08cb-9485-4ff6-9cd7-a8773f489c1d None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:56.618006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee08cb-9485-4ff6-9cd7-a8773f489c1d None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:56.618274 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee08cb-9485-4ff6-9cd7-a8773f489c1d None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:56.619661 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 230/460] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:56.625246 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364dd004-4617-4a2d-890c-e68b7969fef7 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:56.625797 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364dd004-4617-4a2d-890c-e68b7969fef7 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:56.626292 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364dd004-4617-4a2d-890c-e68b7969fef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:56.644989 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-364dd004-4617-4a2d-890c-e68b7969fef7 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:56.646498 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-364dd004-4617-4a2d-890c-e68b7969fef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:56.675506 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:56.721212 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee3cf8-98c5-4a53-913e-97d40ab7aa7a None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:56.727676 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 231/461] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:49:56.727676 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc69b4d-6153-421a-b6bf-e558fa88b95a None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:56.727676 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc69b4d-6153-421a-b6bf-e558fa88b95a None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:56.727676 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc69b4d-6153-421a-b6bf-e558fa88b95a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:56.755587 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-cdc69b4d-6153-421a-b6bf-e558fa88b95a None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:56.756794 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-cdc69b4d-6153-421a-b6bf-e558fa88b95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:56.782262 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516df57c-a902-46f0-97d2-94ccf3ba4ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:56.783268 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516df57c-a902-46f0-97d2-94ccf3ba4ba8 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:56.783820 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516df57c-a902-46f0-97d2-94ccf3ba4ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:56.784880 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516df57c-a902-46f0-97d2-94ccf3ba4ba8 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:56.785954 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516df57c-a902-46f0-97d2-94ccf3ba4ba8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:56.786460 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516df57c-a902-46f0-97d2-94ccf3ba4ba8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:56.788154 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516df57c-a902-46f0-97d2-94ccf3ba4ba8 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:56.803482 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 231/462] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:49:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:56.804703 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516df57c-a902-46f0-97d2-94ccf3ba4ba8 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:56.828060 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:56.835348 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 232/463] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:49:56.835348 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc69b4d-6153-421a-b6bf-e558fa88b95a None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:56.925963 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:56.936494 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc69b4d-6153-421a-b6bf-e558fa88b95a None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:56.937313 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516df57c-a902-46f0-97d2-94ccf3ba4ba8 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:56.972039 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0395c0ea-d116-45f0-b22b-589139e283ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:56.973049 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395c0ea-d116-45f0-b22b-589139e283ae None admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:56.973537 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395c0ea-d116-45f0-b22b-589139e283ae None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:56.974030 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395c0ea-d116-45f0-b22b-589139e283ae None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:56.974599 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6219e2-945f-4960-a201-9d4cf64f2ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:56.975172 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0395c0ea-d116-45f0-b22b-589139e283ae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:56.975444 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6219e2-945f-4960-a201-9d4cf64f2ed3 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:56.975723 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6219e2-945f-4960-a201-9d4cf64f2ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:56.975994 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6219e2-945f-4960-a201-9d4cf64f2ed3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:56.977482 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0395c0ea-d116-45f0-b22b-589139e283ae None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:56.986866 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6219e2-945f-4960-a201-9d4cf64f2ed3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:56.989081 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6219e2-945f-4960-a201-9d4cf64f2ed3 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:57.002865 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 233/464] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Jun 7 21:49:56 2022] POST /identity/v3/users => generated 246 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 21:49:57.004563 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9cd1d-da26-4811-8d8e-367166ddb6c6 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:57.005023 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9cd1d-da26-4811-8d8e-367166ddb6c6 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:57.005465 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9cd1d-da26-4811-8d8e-367166ddb6c6 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:57.007378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 234/465] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:57.008393 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e9cd1d-da26-4811-8d8e-367166ddb6c6 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:57.008597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-7b6219e2-945f-4960-a201-9d4cf64f2ed3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:49:57.008597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:57.008597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:57.008597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:57.008597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:57.008597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:57.008597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:57.008597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:57.008597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:57.008597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:57.008597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:57.008597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:57.008597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:49:57.008597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:49:57.008597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:49:57.009434 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:49:57.009434 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:57.009434 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:57.009434 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:57.009434 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:57.009434 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:57.009434 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:57.009434 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:57.009434 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:57.009434 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:57.009434 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:57.009434 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:57.009434 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:57.009434 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:57.009434 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:57.009434 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:57.009434 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:57.010295 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:49:57.010295 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:49:57.010295 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:49:57.010295 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:49:57.010295 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 232/466] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:49:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:49:57.016989 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6219e2-945f-4960-a201-9d4cf64f2ed3 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:57.099833 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:57.135024 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6219e2-945f-4960-a201-9d4cf64f2ed3 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:57.145772 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e9cd1d-da26-4811-8d8e-367166ddb6c6 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:57.177785 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dec4c99-19ca-42ff-b995-87c9445bd6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:57.178826 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec4c99-19ca-42ff-b995-87c9445bd6a9 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:57.179261 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec4c99-19ca-42ff-b995-87c9445bd6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:57.179741 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec4c99-19ca-42ff-b995-87c9445bd6a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:57.180638 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dec4c99-19ca-42ff-b995-87c9445bd6a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:57.181157 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dec4c99-19ca-42ff-b995-87c9445bd6a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:57.182901 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dec4c99-19ca-42ff-b995-87c9445bd6a9 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:57.187118 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca18070-63b1-459c-bb73-461b9bbd2c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:57.188023 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca18070-63b1-459c-bb73-461b9bbd2c22 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:57.188313 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca18070-63b1-459c-bb73-461b9bbd2c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:57.188592 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca18070-63b1-459c-bb73-461b9bbd2c22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:57.199212 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 233/467] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:49:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:49:57.200931 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff7dd7-d320-4f74-82bc-09cc38b153d8 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:57.201482 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca18070-63b1-459c-bb73-461b9bbd2c22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:57.202333 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff7dd7-d320-4f74-82bc-09cc38b153d8 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:57.202439 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca18070-63b1-459c-bb73-461b9bbd2c22 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:57.203012 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff7dd7-d320-4f74-82bc-09cc38b153d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:57.213479 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:57.224602 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-eca18070-63b1-459c-bb73-461b9bbd2c22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:49:57.224602 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:57.224602 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:57.224602 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:57.224602 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:57.224602 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:57.224602 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:57.224602 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:57.224602 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:57.224602 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:57.224602 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:57.224602 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:57.224602 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:49:57.224602 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:49:57.224602 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:49:57.225527 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:49:57.225527 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:57.225527 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:57.225527 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:57.225527 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:57.225527 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:57.225527 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:57.225527 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:57.225527 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:57.225527 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:57.225527 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:57.225527 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:57.225527 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:57.225527 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:57.225527 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:57.225527 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:57.225527 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:57.226377 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:49:57.226377 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:49:57.226377 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:49:57.226377 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:49:57.227935 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-36ff7dd7-d320-4f74-82bc-09cc38b153d8 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:57.233315 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 235/468] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:49:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:49:57.233315 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca18070-63b1-459c-bb73-461b9bbd2c22 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:57.233577 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-36ff7dd7-d320-4f74-82bc-09cc38b153d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:57.328125 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 234/469] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:49:57.329261 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ff7dd7-d320-4f74-82bc-09cc38b153d8 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:57.358427 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca18070-63b1-459c-bb73-461b9bbd2c22 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:57.400662 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:57.405339 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164c6e14-5f5f-4d15-9d84-f3370c221362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:57.406639 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c6e14-5f5f-4d15-9d84-f3370c221362 None admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:57.407164 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c6e14-5f5f-4d15-9d84-f3370c221362 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:57.407642 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c6e14-5f5f-4d15-9d84-f3370c221362 None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:57.410417 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164c6e14-5f5f-4d15-9d84-f3370c221362 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:57.412552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164c6e14-5f5f-4d15-9d84-f3370c221362 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:57.437399 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 236/470] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Jun 7 21:49:57 2022] POST /identity/v3/users => generated 248 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 21:49:57.438690 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164c6e14-5f5f-4d15-9d84-f3370c221362 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:57.450358 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ff7dd7-d320-4f74-82bc-09cc38b153d8 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:57.512391 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80cc9c1-a337-49fa-bf6d-017ecfd3950a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:57.513086 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80cc9c1-a337-49fa-bf6d-017ecfd3950a None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:57.513371 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80cc9c1-a337-49fa-bf6d-017ecfd3950a None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:57.513648 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80cc9c1-a337-49fa-bf6d-017ecfd3950a None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:57.514401 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80cc9c1-a337-49fa-bf6d-017ecfd3950a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:57.514712 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80cc9c1-a337-49fa-bf6d-017ecfd3950a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:57.516565 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80cc9c1-a337-49fa-bf6d-017ecfd3950a None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:57.527810 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:57.547702 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 235/471] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:49:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:57.554291 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80cc9c1-a337-49fa-bf6d-017ecfd3950a None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:57.576634 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164c6e14-5f5f-4d15-9d84-f3370c221362 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:57.617077 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf603ac3-6d1d-4bc0-9f79-739dc11bcb7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:57.618180 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf603ac3-6d1d-4bc0-9f79-739dc11bcb7e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:57.618648 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf603ac3-6d1d-4bc0-9f79-739dc11bcb7e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:57.619202 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf603ac3-6d1d-4bc0-9f79-739dc11bcb7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:57.632234 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf603ac3-6d1d-4bc0-9f79-739dc11bcb7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:57.634112 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf603ac3-6d1d-4bc0-9f79-739dc11bcb7e None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:57.653916 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-cf603ac3-6d1d-4bc0-9f79-739dc11bcb7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:49:57.653916 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:57.653916 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:57.653916 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:57.653916 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:57.653916 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:57.653916 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:57.653916 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:57.653916 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:57.653916 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:57.653916 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:57.653916 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:57.653916 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:49:57.653916 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:49:57.653916 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:49:57.654841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:49:57.654841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:57.654841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:57.654841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:57.654841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:57.654841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:57.654841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:57.654841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:57.654841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:57.654841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:57.654841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:57.654841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:57.654841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:57.654841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:57.654841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:57.654841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:57.654841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:57.655752 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:49:57.655752 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:49:57.655752 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:49:57.655752 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:49:57.657498 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 237/472] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:49:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:49:57.664829 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf603ac3-6d1d-4bc0-9f79-739dc11bcb7e None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:57.762079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80cc9c1-a337-49fa-bf6d-017ecfd3950a None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:57.803452 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:57.810019 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf603ac3-6d1d-4bc0-9f79-739dc11bcb7e None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:57.819906 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ee286-ffc5-4383-914d-b00fa8c98715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:57.821305 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ee286-ffc5-4383-914d-b00fa8c98715 None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:57.821743 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ee286-ffc5-4383-914d-b00fa8c98715 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:57.822170 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ee286-ffc5-4383-914d-b00fa8c98715 None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:57.823461 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6ee286-ffc5-4383-914d-b00fa8c98715 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:57.825547 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6ee286-ffc5-4383-914d-b00fa8c98715 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:57.827687 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:57.858023 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bee18b-8d08-41bb-b654-6f2b08a73893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:57.858225 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 236/473] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Jun 7 21:49:57 2022] POST /identity/v3/users => generated 248 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 21:49:57.859044 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bee18b-8d08-41bb-b654-6f2b08a73893 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:57.859534 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bee18b-8d08-41bb-b654-6f2b08a73893 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:57.859983 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bee18b-8d08-41bb-b654-6f2b08a73893 None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:57.860244 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dcf855-a005-46c7-8d42-0f32d784e560 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:57.860717 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dcf855-a005-46c7-8d42-0f32d784e560 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:57.861004 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dcf855-a005-46c7-8d42-0f32d784e560 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:57.861516 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bee18b-8d08-41bb-b654-6f2b08a73893 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:57.862005 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bee18b-8d08-41bb-b654-6f2b08a73893 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:57.863297 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 237/474] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:57.864391 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bee18b-8d08-41bb-b654-6f2b08a73893 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:57.870925 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6ef5f-5787-42cc-8e7f-e935fd3f4d0c None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:57.871895 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6ef5f-5787-42cc-8e7f-e935fd3f4d0c None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:57.871895 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6ef5f-5787-42cc-8e7f-e935fd3f4d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:57.887572 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 238/475] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:49:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:57.891779 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-eef6ef5f-5787-42cc-8e7f-e935fd3f4d0c None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:57.893429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bee18b-8d08-41bb-b654-6f2b08a73893 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:57.894511 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-eef6ef5f-5787-42cc-8e7f-e935fd3f4d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:57.992791 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 238/476] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:49:58.028027 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bee18b-8d08-41bb-b654-6f2b08a73893 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:58.035915 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6ef5f-5787-42cc-8e7f-e935fd3f4d0c None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:58.108121 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4304dc9f-2f37-46e7-98c1-11cbbb5c179e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:58.108809 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4304dc9f-2f37-46e7-98c1-11cbbb5c179e None admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:58.109126 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4304dc9f-2f37-46e7-98c1-11cbbb5c179e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:58.109373 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4304dc9f-2f37-46e7-98c1-11cbbb5c179e None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:58.110614 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4304dc9f-2f37-46e7-98c1-11cbbb5c179e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:58.112180 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4304dc9f-2f37-46e7-98c1-11cbbb5c179e None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:58.142715 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 239/477] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Jun 7 21:49:57 2022] POST /identity/v3/users => generated 249 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 21:49:58.196598 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6ef5f-5787-42cc-8e7f-e935fd3f4d0c None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:58.209384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:58.248039 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9f8e8-96a6-4a23-afa4-a42c1d10dd4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:58.249011 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9f8e8-96a6-4a23-afa4-a42c1d10dd4a None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:58.249462 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9f8e8-96a6-4a23-afa4-a42c1d10dd4a None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:58.249876 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9f8e8-96a6-4a23-afa4-a42c1d10dd4a None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:58.250799 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf9f8e8-96a6-4a23-afa4-a42c1d10dd4a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:58.252522 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf9f8e8-96a6-4a23-afa4-a42c1d10dd4a None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:58.262877 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.sql.core [None req-7bf9f8e8-96a6-4a23-afa4-a42c1d10dd4a None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 07 21:49:58.262877 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 07 21:49:58.262877 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [parameters: {'id': 'a2940a4abde34a37b46d81ff61d18ce0', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 07 21:49:58.262877 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80445) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 07 21:49:58.281936 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-7bf9f8e8-96a6-4a23-afa4-a42c1d10dd4a None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 07 21:49:58.281936 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:58.281936 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 21:49:58.281936 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 21:49:58.281936 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 21:49:58.281936 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 21:49:58.281936 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 21:49:58.281936 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 21:49:58.281936 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 21:49:58.281936 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application conn.query(q) Jun 07 21:49:58.281936 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 21:49:58.281936 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 21:49:58.281936 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 21:49:58.281936 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application result.read() Jun 07 21:49:58.281936 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 21:49:58.282998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 21:49:58.282998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 21:49:58.282998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 21:49:58.282998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 21:49:58.282998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 21:49:58.282998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 21:49:58.282998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 21:49:58.282998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 07 21:49:58.282998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:49:58.282998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 07 21:49:58.282998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:49:58.282998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:58.282998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 07 21:49:58.282998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 07 21:49:58.282998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 07 21:49:58.282998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return ref.to_dict() Jun 07 21:49:58.282998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 21:49:58.282998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application next(self.gen) Jun 07 21:49:58.284129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 07 21:49:58.284129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application yield resource Jun 07 21:49:58.284129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 21:49:58.284129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application next(self.gen) Jun 07 21:49:58.284129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 07 21:49:58.284129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self.session.rollback() Jun 07 21:49:58.284129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 07 21:49:58.284129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self.force_reraise() Jun 07 21:49:58.284129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 07 21:49:58.284129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise self.value Jun 07 21:49:58.284129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 07 21:49:58.284129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 07 21:49:58.284129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 07 21:49:58.284129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application session.commit() Jun 07 21:49:58.284129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 07 21:49:58.284129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 07 21:49:58.284129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 07 21:49:58.285861 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._prepare_impl() Jun 07 21:49:58.285861 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 07 21:49:58.285861 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self.session.flush() Jun 07 21:49:58.285861 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 07 21:49:58.285861 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._flush(objects) Jun 07 21:49:58.285861 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 07 21:49:58.285861 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 07 21:49:58.285861 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 07 21:49:58.285861 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application compat.raise_( Jun 07 21:49:58.285861 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 21:49:58.285861 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception Jun 07 21:49:58.285861 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 07 21:49:58.285861 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application flush_context.execute() Jun 07 21:49:58.285861 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 07 21:49:58.285861 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rec.execute(self) Jun 07 21:49:58.285861 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 07 21:49:58.285861 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 07 21:49:58.286751 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 07 21:49:58.286751 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 07 21:49:58.286751 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 07 21:49:58.286751 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 07 21:49:58.286751 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 07 21:49:58.286751 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 07 21:49:58.286751 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 07 21:49:58.286751 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 07 21:49:58.286751 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 07 21:49:58.286751 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 07 21:49:58.286751 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 07 21:49:58.286751 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 07 21:49:58.286751 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 07 21:49:58.286751 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 07 21:49:58.286751 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 21:49:58.286751 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception Jun 07 21:49:58.287717 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 21:49:58.287717 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 21:49:58.287717 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 21:49:58.287717 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 21:49:58.287717 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 21:49:58.287717 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 21:49:58.287717 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 21:49:58.287717 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application conn.query(q) Jun 07 21:49:58.287717 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 21:49:58.287717 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 21:49:58.287717 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 21:49:58.287717 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application result.read() Jun 07 21:49:58.287717 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 21:49:58.287717 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 21:49:58.287717 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 21:49:58.287717 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 21:49:58.288501 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 21:49:58.288501 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 21:49:58.288501 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 21:49:58.288501 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 21:49:58.288501 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 07 21:49:58.288501 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 07 21:49:58.288501 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [parameters: {'id': 'a2940a4abde34a37b46d81ff61d18ce0', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 07 21:49:58.288501 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 07 21:49:58.288501 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:49:58.288501 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 21:49:58.288501 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:49:58.288501 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:58.288501 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:58.288501 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:58.288501 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:58.288501 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:58.289316 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:58.289316 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:58.289316 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:58.289316 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:58.289316 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:58.289316 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:58.289316 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 07 21:49:58.289316 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 07 21:49:58.289316 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:58.289316 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:58.289316 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 07 21:49:58.289316 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 07 21:49:58.289316 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 07 21:49:58.289316 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 07 21:49:58.289316 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 07 21:49:58.289316 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:49:58.292205 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 239/478] 10.222.0.13 () {60 vars in 1131 bytes} [Tue Jun 7 21:49:58 2022] POST /identity/v3/roles => generated 144 bytes in 257 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:49:58.298349 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4304dc9f-2f37-46e7-98c1-11cbbb5c179e None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:58.340916 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:58.429801 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4304dc9f-2f37-46e7-98c1-11cbbb5c179e None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:58.468454 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a22807a-21bc-465b-8170-e12886802e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:58.469271 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22807a-21bc-465b-8170-e12886802e4f None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:58.469554 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22807a-21bc-465b-8170-e12886802e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:58.469828 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22807a-21bc-465b-8170-e12886802e4f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:58.481251 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a22807a-21bc-465b-8170-e12886802e4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:49:58.491775 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-3a22807a-21bc-465b-8170-e12886802e4f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 21:49:58.491775 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:49:58.491775 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:49:58.491775 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:49:58.491775 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:49:58.491775 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:49:58.491775 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:49:58.491775 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:49:58.491775 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:49:58.491775 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:49:58.491775 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:49:58.491775 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:49:58.491775 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:49:58.491775 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:49:58.491775 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:49:58.491775 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:49:58.492708 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:49:58.492708 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:49:58.492708 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:49:58.492708 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:49:58.492708 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:49:58.492708 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:49:58.492708 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:49:58.492708 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:49:58.492708 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:49:58.492708 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:49:58.492708 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:49:58.492708 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:49:58.492708 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:49:58.492708 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:49:58.492708 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:49:58.492708 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:49:58.493523 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:49:58.493523 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:49:58.493523 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:49:58.493523 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:49:58.493523 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:49:58.493523 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:49:58.493523 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:49:58.493523 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:49:58.493523 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:49:58.493523 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:49:58.493523 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 21:49:58.493523 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:49:58.494141 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 240/479] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Jun 7 21:49:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:49:58.500305 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9f8e8-96a6-4a23-afa4-a42c1d10dd4a None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:49:58.644358 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9f8e8-96a6-4a23-afa4-a42c1d10dd4a None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:49:58.688467 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68166af-25af-4c98-8f73-89e10ba8ca1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:49:58.689120 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68166af-25af-4c98-8f73-89e10ba8ca1f None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:58.689408 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68166af-25af-4c98-8f73-89e10ba8ca1f None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:58.689686 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68166af-25af-4c98-8f73-89e10ba8ca1f None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:58.690506 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68166af-25af-4c98-8f73-89e10ba8ca1f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:49:58.690820 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68166af-25af-4c98-8f73-89e10ba8ca1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:49:58.692243 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68166af-25af-4c98-8f73-89e10ba8ca1f None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:49:58.707825 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 240/480] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:49:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:49:59.813544 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9132db78-d94d-4a91-bf0c-cfeea20f7929 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:59.814515 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9132db78-d94d-4a91-bf0c-cfeea20f7929 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:59.814987 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9132db78-d94d-4a91-bf0c-cfeea20f7929 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:59.817401 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 241/481] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:49:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:49:59.822951 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077944f6-dd02-4a61-b096-d2b2d4159969 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:49:59.823798 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077944f6-dd02-4a61-b096-d2b2d4159969 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:49:59.824278 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077944f6-dd02-4a61-b096-d2b2d4159969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:49:59.852335 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-077944f6-dd02-4a61-b096-d2b2d4159969 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:49:59.853878 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-077944f6-dd02-4a61-b096-d2b2d4159969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:49:59.888665 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:49:59.944235 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 241/482] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:49:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:49:59.989199 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9132db78-d94d-4a91-bf0c-cfeea20f7929 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:00.218231 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:00.243889 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9132db78-d94d-4a91-bf0c-cfeea20f7929 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:00.291355 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79100b77-2fde-4e70-acc1-1636f51ec9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:00.292145 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79100b77-2fde-4e70-acc1-1636f51ec9f1 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:00.292420 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79100b77-2fde-4e70-acc1-1636f51ec9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:00.292689 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79100b77-2fde-4e70-acc1-1636f51ec9f1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:00.302724 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79100b77-2fde-4e70-acc1-1636f51ec9f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:50:00.310291 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-79100b77-2fde-4e70-acc1-1636f51ec9f1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 21:50:00.310291 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:00.310291 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:00.310291 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:00.310291 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:00.310291 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:00.310291 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:00.310291 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:00.310291 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:00.310291 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:00.310291 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:00.310291 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:00.310291 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:50:00.310291 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:50:00.310291 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:50:00.310291 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:50:00.311153 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:50:00.311153 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:50:00.311153 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:50:00.311153 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:50:00.311153 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:00.311153 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:00.311153 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:00.311153 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:00.311153 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:00.311153 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:00.311153 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:00.311153 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:00.311153 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:00.311153 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:00.311153 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:00.311153 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:00.311995 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:00.311995 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:00.311995 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:00.311995 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:00.311995 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:50:00.311995 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:50:00.311995 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:50:00.311995 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:50:00.311995 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:50:00.311995 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:50:00.311995 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 21:50:00.311995 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:00.315737 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 242/483] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jun 7 21:49:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:00.318188 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077944f6-dd02-4a61-b096-d2b2d4159969 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:00.459266 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077944f6-dd02-4a61-b096-d2b2d4159969 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:00.514871 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4905e73b-fd1e-479d-be04-4f107c46e15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:00.515516 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905e73b-fd1e-479d-be04-4f107c46e15a None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:00.515806 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905e73b-fd1e-479d-be04-4f107c46e15a None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:00.516079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905e73b-fd1e-479d-be04-4f107c46e15a None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:00.516861 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4905e73b-fd1e-479d-be04-4f107c46e15a None admin] RBAC: Adding query filter params (name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:00.517165 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4905e73b-fd1e-479d-be04-4f107c46e15a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:00.519271 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4905e73b-fd1e-479d-be04-4f107c46e15a None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:00.532250 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 242/484] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Jun 7 21:50:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:00.538123 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79100b77-2fde-4e70-acc1-1636f51ec9f1 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:00.662242 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a4693-1039-44aa-bc6b-3dabdecca3d7 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:00.662623 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a4693-1039-44aa-bc6b-3dabdecca3d7 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:00.662801 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a4693-1039-44aa-bc6b-3dabdecca3d7 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:00.665515 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 243/485] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 21:50:00.669630 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2ac97-0c88-4c13-8ad0-5812748af18c None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:00.669921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2ac97-0c88-4c13-8ad0-5812748af18c None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:00.670203 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2ac97-0c88-4c13-8ad0-5812748af18c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:00.687927 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-a4c2ac97-0c88-4c13-8ad0-5812748af18c None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:00.689408 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-a4c2ac97-0c88-4c13-8ad0-5812748af18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:00.700144 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79100b77-2fde-4e70-acc1-1636f51ec9f1 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:00.713539 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:00.779535 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb55d9c-fed6-4583-8dc4-3ab72b7136aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:00.780260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb55d9c-fed6-4583-8dc4-3ab72b7136aa None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:00.780548 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb55d9c-fed6-4583-8dc4-3ab72b7136aa None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:00.780829 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb55d9c-fed6-4583-8dc4-3ab72b7136aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:00.794642 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 244/486] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:50:00.796845 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb55d9c-fed6-4583-8dc4-3ab72b7136aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:00.798434 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb55d9c-fed6-4583-8dc4-3ab72b7136aa None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:00.806271 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:00.820020 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c2ac97-0c88-4c13-8ad0-5812748af18c None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:00.835345 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-3eb55d9c-fed6-4583-8dc4-3ab72b7136aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:00.835345 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:00.835345 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:00.835345 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:00.835345 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:00.835345 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:00.835345 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:00.835345 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:00.835345 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:00.835345 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:00.835345 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:00.835345 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:00.835345 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:00.835345 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:00.835345 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:00.836552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:00.836552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:00.836552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:00.836552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:00.836552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:00.836552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:00.836552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:00.836552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:00.836552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:00.836552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:00.836552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:00.836552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:00.836552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:00.836552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:00.836552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:00.836552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:00.836552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:00.837483 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:00.837483 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:00.837483 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:00.837483 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:00.838143 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 243/487] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:00.843731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb55d9c-fed6-4583-8dc4-3ab72b7136aa None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:01.000828 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c2ac97-0c88-4c13-8ad0-5812748af18c None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:01.004150 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb55d9c-fed6-4583-8dc4-3ab72b7136aa None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:01.044897 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3831595-4e85-497f-bf2d-74000a3c2fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:01.045653 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3831595-4e85-497f-bf2d-74000a3c2fd8 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:01.046003 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3831595-4e85-497f-bf2d-74000a3c2fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:01.046368 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3831595-4e85-497f-bf2d-74000a3c2fd8 None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:01.047163 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3831595-4e85-497f-bf2d-74000a3c2fd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:01.047578 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3831595-4e85-497f-bf2d-74000a3c2fd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:01.049183 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3831595-4e85-497f-bf2d-74000a3c2fd8 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:01.049408 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738650e7-f554-4680-ac45-37546a4f4f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:01.050974 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738650e7-f554-4680-ac45-37546a4f4f42 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:01.051800 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738650e7-f554-4680-ac45-37546a4f4f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:01.052253 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738650e7-f554-4680-ac45-37546a4f4f42 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:01.065906 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738650e7-f554-4680-ac45-37546a4f4f42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:50:01.066491 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 244/488] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:01.067896 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237866f7-f60c-4df4-a2ec-3e6d20006967 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:01.068183 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237866f7-f60c-4df4-a2ec-3e6d20006967 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:01.068460 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237866f7-f60c-4df4-a2ec-3e6d20006967 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:01.071432 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 245/489] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:01.071432 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237866f7-f60c-4df4-a2ec-3e6d20006967 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:01.080622 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-738650e7-f554-4680-ac45-37546a4f4f42 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 21:50:01.080622 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:01.080622 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:01.080622 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:01.080622 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:01.080622 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:01.080622 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:01.080622 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:01.080622 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:01.080622 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:01.080622 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:01.080622 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:01.080622 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:50:01.080622 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:50:01.080622 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:50:01.080622 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:50:01.081720 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:50:01.081720 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:50:01.081720 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:50:01.081720 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:50:01.081720 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:01.081720 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:01.081720 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:01.081720 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:01.081720 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:01.081720 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:01.081720 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:01.081720 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:01.081720 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:01.081720 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:01.081720 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:01.081720 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:01.082626 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:01.082626 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:01.082626 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:01.082626 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:01.082626 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:50:01.082626 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:50:01.082626 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:50:01.082626 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:50:01.082626 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:50:01.082626 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:50:01.082626 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 21:50:01.082626 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:01.088102 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 245/490] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jun 7 21:50:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:50:01.088102 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828cc33f-1d33-4ecb-85eb-6854277a25d9 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:01.088102 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828cc33f-1d33-4ecb-85eb-6854277a25d9 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:01.088102 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828cc33f-1d33-4ecb-85eb-6854277a25d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:01.107112 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-828cc33f-1d33-4ecb-85eb-6854277a25d9 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:01.108407 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-828cc33f-1d33-4ecb-85eb-6854277a25d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:01.121826 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:01.144210 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:01.192863 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237866f7-f60c-4df4-a2ec-3e6d20006967 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:01.199431 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 246/491] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:50:01.200360 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828cc33f-1d33-4ecb-85eb-6854277a25d9 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:01.229262 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbdfcfa-eb0d-4bf2-b67e-3d81a0bfef75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:01.230298 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbdfcfa-eb0d-4bf2-b67e-3d81a0bfef75 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:01.230716 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbdfcfa-eb0d-4bf2-b67e-3d81a0bfef75 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:01.231123 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbdfcfa-eb0d-4bf2-b67e-3d81a0bfef75 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:01.243599 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbdfcfa-eb0d-4bf2-b67e-3d81a0bfef75 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:01.245349 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbdfcfa-eb0d-4bf2-b67e-3d81a0bfef75 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:01.265330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-dcbdfcfa-eb0d-4bf2-b67e-3d81a0bfef75 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 21:50:01.265330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:01.265330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:01.265330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:01.265330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:01.265330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:01.265330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:01.265330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:01.265330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:01.265330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:01.265330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:01.265330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:01.265330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 21:50:01.265330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 21:50:01.265330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 21:50:01.265330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 21:50:01.266495 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:01.266495 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:01.266495 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 21:50:01.266495 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:01.266495 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 21:50:01.266495 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:01.266495 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:01.266495 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:01.266495 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:01.266495 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:01.266495 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:01.266495 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:01.266495 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:01.266495 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:01.266495 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:01.266495 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:01.266495 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:01.266495 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:01.267537 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:01.267537 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:01.267537 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 21:50:01.267537 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 21:50:01.267537 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 21:50:01.267537 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 21:50:01.267537 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 21:50:01.267537 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 21:50:01.267537 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 21:50:01.267537 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:01.269603 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 246/492] 10.222.0.13 () {56 vars in 1093 bytes} [Tue Jun 7 21:50:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:50:01.274879 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbdfcfa-eb0d-4bf2-b67e-3d81a0bfef75 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:01.353164 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828cc33f-1d33-4ecb-85eb-6854277a25d9 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:01.386207 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbdfcfa-eb0d-4bf2-b67e-3d81a0bfef75 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:01.394391 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c8cea4-e268-423b-a1fd-fb08a2e527df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:01.395072 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8cea4-e268-423b-a1fd-fb08a2e527df None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:01.395396 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8cea4-e268-423b-a1fd-fb08a2e527df None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:01.395680 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8cea4-e268-423b-a1fd-fb08a2e527df None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:01.396437 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c8cea4-e268-423b-a1fd-fb08a2e527df None admin] RBAC: Adding query filter params (name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:01.396744 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c8cea4-e268-423b-a1fd-fb08a2e527df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:01.398107 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c8cea4-e268-423b-a1fd-fb08a2e527df None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:01.403923 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:01.406938 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 247/493] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Jun 7 21:50:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:01.407986 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c8cea4-e268-423b-a1fd-fb08a2e527df None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:01.422827 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:01.426455 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a308fc5-9cdc-4972-8786-c0bcfdb8af4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:01.429068 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a308fc5-9cdc-4972-8786-c0bcfdb8af4c None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:01.429358 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a308fc5-9cdc-4972-8786-c0bcfdb8af4c None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:01.429703 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a308fc5-9cdc-4972-8786-c0bcfdb8af4c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:01.441392 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a308fc5-9cdc-4972-8786-c0bcfdb8af4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:50:01.449084 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-5a308fc5-9cdc-4972-8786-c0bcfdb8af4c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 21:50:01.449084 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:01.449084 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:01.449084 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:01.449084 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:01.449084 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:01.449084 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:01.449084 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:01.449084 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:01.449084 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:01.449084 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:01.449084 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:01.449084 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:50:01.449084 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:50:01.449084 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:50:01.449084 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:50:01.449953 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:50:01.449953 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:50:01.449953 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:50:01.449953 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:50:01.449953 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:01.449953 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:01.449953 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:01.449953 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:01.449953 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:01.449953 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:01.449953 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:01.449953 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:01.449953 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:01.449953 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:01.449953 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:01.449953 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:01.450765 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:01.450765 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:01.450765 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:01.450765 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:01.450765 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:50:01.450765 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:50:01.450765 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:50:01.450765 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:50:01.450765 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:50:01.450765 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:50:01.450765 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 21:50:01.450765 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:01.450765 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 247/494] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jun 7 21:50:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:01.451968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bcc092-a266-4a4b-ba59-225a0b049b0c None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:01.451968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bcc092-a266-4a4b-ba59-225a0b049b0c None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:01.452106 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bcc092-a266-4a4b-ba59-225a0b049b0c None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:01.453424 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 248/495] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:01.456854 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bcc092-a266-4a4b-ba59-225a0b049b0c None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:01.529396 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c8cea4-e268-423b-a1fd-fb08a2e527df None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:01.567947 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bcc092-a266-4a4b-ba59-225a0b049b0c None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:01.570111 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464602d9-d150-45ca-96a8-8a559b9accfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:01.572245 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464602d9-d150-45ca-96a8-8a559b9accfe None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:01.572747 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464602d9-d150-45ca-96a8-8a559b9accfe None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:01.573169 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464602d9-d150-45ca-96a8-8a559b9accfe None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:01.574776 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464602d9-d150-45ca-96a8-8a559b9accfe None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:01.575940 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464602d9-d150-45ca-96a8-8a559b9accfe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:01.579057 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464602d9-d150-45ca-96a8-8a559b9accfe None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:01.604704 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b6a38c-96c2-4416-b97e-00702187c0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:01.605417 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6a38c-96c2-4416-b97e-00702187c0b8 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:01.605696 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6a38c-96c2-4416-b97e-00702187c0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:01.605968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6a38c-96c2-4416-b97e-00702187c0b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:01.617657 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b6a38c-96c2-4416-b97e-00702187c0b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:01.619133 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b6a38c-96c2-4416-b97e-00702187c0b8 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:01.619803 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 248/496] 10.222.0.13 () {56 vars in 1133 bytes} [Tue Jun 7 21:50:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:01.620674 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464602d9-d150-45ca-96a8-8a559b9accfe None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:01.644230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-b2b6a38c-96c2-4416-b97e-00702187c0b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:01.644230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:01.644230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:01.644230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:01.644230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:01.644230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:01.644230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:01.644230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:01.644230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:01.644230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:01.644230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:01.644230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:01.644230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:01.644230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:01.644230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:01.645429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:01.645429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:01.645429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:01.645429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:01.645429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:01.645429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:01.645429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:01.645429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:01.645429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:01.645429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:01.645429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:01.645429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:01.645429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:01.645429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:01.645429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:01.645429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:01.645429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:01.646453 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:01.646453 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:01.646453 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:01.646453 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:01.646453 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 249/497] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:01.648567 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5323d33-0e6c-41ce-a6ad-f9b8fac4ce97 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:01.649076 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5323d33-0e6c-41ce-a6ad-f9b8fac4ce97 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:01.649556 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5323d33-0e6c-41ce-a6ad-f9b8fac4ce97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:01.670348 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-a5323d33-0e6c-41ce-a6ad-f9b8fac4ce97 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:01.671669 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a5323d33-0e6c-41ce-a6ad-f9b8fac4ce97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:01.694963 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:01.697787 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:01.729765 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464602d9-d150-45ca-96a8-8a559b9accfe None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:01.750073 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 250/498] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:50:01.750924 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5323d33-0e6c-41ce-a6ad-f9b8fac4ce97 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:01.767905 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4de109-d642-4d53-b614-cafbe27a8d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:01.768703 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4de109-d642-4d53-b614-cafbe27a8d6e None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:01.768991 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4de109-d642-4d53-b614-cafbe27a8d6e None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:01.769370 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4de109-d642-4d53-b614-cafbe27a8d6e None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:01.770229 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4de109-d642-4d53-b614-cafbe27a8d6e None admin] RBAC: Adding query filter params (name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:01.770541 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4de109-d642-4d53-b614-cafbe27a8d6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:01.772034 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4de109-d642-4d53-b614-cafbe27a8d6e None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:01.780763 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 249/499] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Jun 7 21:50:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:01.781628 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4de109-d642-4d53-b614-cafbe27a8d6e None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:01.856461 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5323d33-0e6c-41ce-a6ad-f9b8fac4ce97 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:01.881972 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4de109-d642-4d53-b614-cafbe27a8d6e None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:01.889837 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b18d79f-ba46-47e3-8174-c2c394304d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:01.890486 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b18d79f-ba46-47e3-8174-c2c394304d58 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:01.890771 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b18d79f-ba46-47e3-8174-c2c394304d58 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:01.891108 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b18d79f-ba46-47e3-8174-c2c394304d58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:01.907084 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:01.915761 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7497f75a-b496-4666-bef8-fa874442e228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:01.916422 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7497f75a-b496-4666-bef8-fa874442e228 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:01.916708 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7497f75a-b496-4666-bef8-fa874442e228 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:01.916983 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7497f75a-b496-4666-bef8-fa874442e228 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:01.917703 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7497f75a-b496-4666-bef8-fa874442e228 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:01.917801 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:01.918008 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7497f75a-b496-4666-bef8-fa874442e228 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:01.919882 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7497f75a-b496-4666-bef8-fa874442e228 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:01.960479 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b18d79f-ba46-47e3-8174-c2c394304d58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:01.961958 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b18d79f-ba46-47e3-8174-c2c394304d58 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:02.053162 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 250/500] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:02.054489 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad7894-a975-4a1c-a9a5-d0aa9cbc2824 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:02.054780 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad7894-a975-4a1c-a9a5-d0aa9cbc2824 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:02.055059 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad7894-a975-4a1c-a9a5-d0aa9cbc2824 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:02.057839 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 251/501] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:02.057839 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad7894-a975-4a1c-a9a5-d0aa9cbc2824 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:02.074376 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 251/502] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:50:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 21:50:02.074376 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b18d79f-ba46-47e3-8174-c2c394304d58 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:02.132049 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:02.158773 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad7894-a975-4a1c-a9a5-d0aa9cbc2824 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:02.172866 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b18d79f-ba46-47e3-8174-c2c394304d58 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:02.192788 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e02bb9-c598-4662-89b6-e55008699a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:02.193582 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e02bb9-c598-4662-89b6-e55008699a8a None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:02.193865 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e02bb9-c598-4662-89b6-e55008699a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:02.194140 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e02bb9-c598-4662-89b6-e55008699a8a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:02.204074 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e02bb9-c598-4662-89b6-e55008699a8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:50:02.207674 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee0154-db29-4bd3-aa27-247c151b217d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:02.208462 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee0154-db29-4bd3-aa27-247c151b217d None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:02.208746 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee0154-db29-4bd3-aa27-247c151b217d None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:02.209024 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee0154-db29-4bd3-aa27-247c151b217d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:02.212390 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-d4e02bb9-c598-4662-89b6-e55008699a8a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 21:50:02.212390 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:02.212390 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:02.212390 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:02.212390 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:02.212390 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:02.212390 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:02.212390 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:02.212390 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:02.212390 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:02.212390 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:02.212390 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:02.212390 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:50:02.212390 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:50:02.212390 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:50:02.212390 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:50:02.213464 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:50:02.213464 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:50:02.213464 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:50:02.213464 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:50:02.213464 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:02.213464 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:02.213464 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:02.213464 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:02.213464 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:02.213464 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:02.213464 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:02.213464 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:02.213464 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:02.213464 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:02.213464 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:02.213464 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:02.219172 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:02.219172 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:02.219172 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:02.219172 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:02.219172 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:50:02.219172 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:50:02.219172 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:50:02.219172 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:50:02.219172 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:50:02.219172 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:50:02.219172 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 21:50:02.219172 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:02.219172 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 252/503] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jun 7 21:50:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:02.219172 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e02bb9-c598-4662-89b6-e55008699a8a None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:02.224760 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dee0154-db29-4bd3-aa27-247c151b217d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:02.226225 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dee0154-db29-4bd3-aa27-247c151b217d None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:02.249108 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-3dee0154-db29-4bd3-aa27-247c151b217d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:02.249108 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:02.249108 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:02.249108 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:02.249108 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:02.249108 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:02.249108 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:02.249108 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:02.249108 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:02.249108 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:02.249108 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:02.249108 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:02.249108 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:02.249108 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:02.249108 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:02.250023 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:02.250023 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:02.250023 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:02.250023 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:02.250023 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:02.250023 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:02.250023 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:02.250023 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:02.250023 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:02.250023 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:02.250023 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:02.250023 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:02.250023 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:02.250023 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:02.250023 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:02.250023 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:02.250023 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:02.250853 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:02.250853 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:02.250853 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:02.250853 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:02.250853 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 252/504] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:02.251517 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a9b221-b0dd-4835-9689-567a88ebf3c1 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:02.251784 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a9b221-b0dd-4835-9689-567a88ebf3c1 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:02.252061 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a9b221-b0dd-4835-9689-567a88ebf3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:02.269290 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-e6a9b221-b0dd-4835-9689-567a88ebf3c1 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:02.269801 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:02.270484 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-e6a9b221-b0dd-4835-9689-567a88ebf3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:02.320827 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e02bb9-c598-4662-89b6-e55008699a8a None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:02.336986 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 253/505] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:50:02.337951 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a9b221-b0dd-4835-9689-567a88ebf3c1 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:02.356494 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb47feed-c5cb-440a-9a5b-f86980729d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:02.357795 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb47feed-c5cb-440a-9a5b-f86980729d55 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:02.358262 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb47feed-c5cb-440a-9a5b-f86980729d55 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:02.358716 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb47feed-c5cb-440a-9a5b-f86980729d55 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:02.367453 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb47feed-c5cb-440a-9a5b-f86980729d55 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:02.368891 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb47feed-c5cb-440a-9a5b-f86980729d55 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:02.380106 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:02.384482 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-cb47feed-c5cb-440a-9a5b-f86980729d55 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 07 21:50:02.384482 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:02.384482 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:02.384482 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:02.384482 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:02.384482 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:02.384482 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:02.384482 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:02.384482 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:02.384482 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:02.384482 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:02.384482 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:02.384482 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 21:50:02.384482 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 21:50:02.384482 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 21:50:02.384482 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 21:50:02.385421 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:02.385421 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:02.385421 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 21:50:02.385421 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:02.385421 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 21:50:02.385421 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:02.385421 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:02.385421 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:02.385421 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:02.385421 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:02.385421 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:02.385421 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:02.385421 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:02.385421 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:02.385421 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:02.385421 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:02.385421 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:02.385421 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:02.386336 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:02.386336 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:02.386336 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 21:50:02.386336 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 21:50:02.386336 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 21:50:02.386336 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 21:50:02.386336 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 21:50:02.386336 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 21:50:02.386336 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 07 21:50:02.386336 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:02.386336 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 253/506] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:50:02 2022] GET /identity/v3/users/glance => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:02.386916 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb47feed-c5cb-440a-9a5b-f86980729d55 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:02.439834 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a9b221-b0dd-4835-9689-567a88ebf3c1 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:02.474094 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dea6a9b-ccc4-4aeb-ae02-cf3f217ba040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:02.474887 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea6a9b-ccc4-4aeb-ae02-cf3f217ba040 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:02.475168 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea6a9b-ccc4-4aeb-ae02-cf3f217ba040 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:02.475482 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea6a9b-ccc4-4aeb-ae02-cf3f217ba040 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:02.485552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dea6a9b-ccc4-4aeb-ae02-cf3f217ba040 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:02.487023 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dea6a9b-ccc4-4aeb-ae02-cf3f217ba040 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:02.489557 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb47feed-c5cb-440a-9a5b-f86980729d55 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:02.498133 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:02.505737 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-9dea6a9b-ccc4-4aeb-ae02-cf3f217ba040 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:02.505737 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:02.505737 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:02.505737 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:02.505737 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:02.505737 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:02.505737 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:02.505737 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:02.505737 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:02.505737 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:02.505737 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:02.505737 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:02.505737 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:02.505737 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:02.505737 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:02.506597 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:02.506597 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:02.506597 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:02.506597 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:02.506597 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:02.506597 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:02.506597 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:02.506597 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:02.506597 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:02.506597 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:02.506597 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:02.506597 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:02.506597 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:02.506597 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:02.506597 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:02.506597 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:02.506597 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:02.507744 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:02.507744 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:02.507744 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:02.507744 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:02.507744 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 254/507] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:02.508028 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dea6a9b-ccc4-4aeb-ae02-cf3f217ba040 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:02.524916 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bc78c1-2e31-4eb1-8723-aedf72b651ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:02.525606 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc78c1-2e31-4eb1-8723-aedf72b651ee None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:02.525895 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc78c1-2e31-4eb1-8723-aedf72b651ee None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:02.526172 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc78c1-2e31-4eb1-8723-aedf72b651ee None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:02.526926 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bc78c1-2e31-4eb1-8723-aedf72b651ee None admin] RBAC: Adding query filter params (name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:02.527228 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bc78c1-2e31-4eb1-8723-aedf72b651ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:02.528630 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bc78c1-2e31-4eb1-8723-aedf72b651ee None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:02.536934 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 254/508] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Jun 7 21:50:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:02.537758 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bc78c1-2e31-4eb1-8723-aedf72b651ee None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:02.607351 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:02.616554 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dea6a9b-ccc4-4aeb-ae02-cf3f217ba040 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:02.650059 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bc78c1-2e31-4eb1-8723-aedf72b651ee None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:02.655598 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e843414b-a8bb-4765-be62-80a22c982667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:02.656263 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e843414b-a8bb-4765-be62-80a22c982667 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:02.656547 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e843414b-a8bb-4765-be62-80a22c982667 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:02.656836 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e843414b-a8bb-4765-be62-80a22c982667 None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:02.657563 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e843414b-a8bb-4765-be62-80a22c982667 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:02.657866 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e843414b-a8bb-4765-be62-80a22c982667 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:02.659316 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e843414b-a8bb-4765-be62-80a22c982667 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:02.670491 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 255/509] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:02.671355 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e843414b-a8bb-4765-be62-80a22c982667 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:02.684657 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b96073c-ed6f-4d7d-a064-c74f5b39e5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:02.685457 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96073c-ed6f-4d7d-a064-c74f5b39e5f8 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:02.685741 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96073c-ed6f-4d7d-a064-c74f5b39e5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:02.686016 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96073c-ed6f-4d7d-a064-c74f5b39e5f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:02.697713 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b96073c-ed6f-4d7d-a064-c74f5b39e5f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:02.699143 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b96073c-ed6f-4d7d-a064-c74f5b39e5f8 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:02.716887 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-2b96073c-ed6f-4d7d-a064-c74f5b39e5f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:02.716887 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:02.716887 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:02.716887 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:02.716887 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:02.716887 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:02.716887 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:02.716887 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:02.716887 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:02.716887 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:02.716887 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:02.716887 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:02.716887 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:02.716887 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:02.716887 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:02.717731 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:02.717731 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:02.717731 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:02.717731 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:02.717731 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:02.717731 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:02.717731 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:02.717731 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:02.717731 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:02.717731 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:02.717731 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:02.717731 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:02.717731 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:02.717731 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:02.717731 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:02.717731 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:02.717731 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:02.718572 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:02.718572 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:02.718572 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:02.718572 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:02.718572 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 255/510] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:02.719215 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b96073c-ed6f-4d7d-a064-c74f5b39e5f8 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:02.743971 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:02.778718 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e843414b-a8bb-4765-be62-80a22c982667 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:02.811142 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb7407d-8560-4c0c-ae27-65cd7d5c285e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:02.811839 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7407d-8560-4c0c-ae27-65cd7d5c285e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:02.812123 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7407d-8560-4c0c-ae27-65cd7d5c285e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:02.812396 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7407d-8560-4c0c-ae27-65cd7d5c285e None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:02.813224 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb7407d-8560-4c0c-ae27-65cd7d5c285e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:02.813533 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb7407d-8560-4c0c-ae27-65cd7d5c285e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:02.814960 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb7407d-8560-4c0c-ae27-65cd7d5c285e None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:02.820660 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b96073c-ed6f-4d7d-a064-c74f5b39e5f8 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:02.851959 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 256/511] 10.222.0.13 () {56 vars in 1137 bytes} [Tue Jun 7 21:50:02 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:02.852995 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb7407d-8560-4c0c-ae27-65cd7d5c285e None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:02.857925 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e24556-47a8-42ed-9f13-28ee333a795e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:02.858595 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e24556-47a8-42ed-9f13-28ee333a795e None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:02.858882 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e24556-47a8-42ed-9f13-28ee333a795e None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:02.859164 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e24556-47a8-42ed-9f13-28ee333a795e None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:02.859928 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e24556-47a8-42ed-9f13-28ee333a795e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:02.860240 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e24556-47a8-42ed-9f13-28ee333a795e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:02.861653 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e24556-47a8-42ed-9f13-28ee333a795e None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:02.866365 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:02.874642 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 256/512] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:02.875486 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e24556-47a8-42ed-9f13-28ee333a795e None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:02.965998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:02.966791 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb7407d-8560-4c0c-ae27-65cd7d5c285e None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:02.984601 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e24556-47a8-42ed-9f13-28ee333a795e None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:03.002027 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9944c796-8729-4310-922b-8a8c2bf89942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:03.003434 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9944c796-8729-4310-922b-8a8c2bf89942 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:03.003763 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9944c796-8729-4310-922b-8a8c2bf89942 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:03.003990 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9944c796-8729-4310-922b-8a8c2bf89942 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:03.013710 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9944c796-8729-4310-922b-8a8c2bf89942 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:03.015139 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9944c796-8729-4310-922b-8a8c2bf89942 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:03.018708 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07983622-3a0e-42f3-822d-2f7b1bc92f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:03.019535 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07983622-3a0e-42f3-822d-2f7b1bc92f5f None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:03.019824 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07983622-3a0e-42f3-822d-2f7b1bc92f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:03.020101 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07983622-3a0e-42f3-822d-2f7b1bc92f5f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:03.027096 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07983622-3a0e-42f3-822d-2f7b1bc92f5f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:03.028560 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07983622-3a0e-42f3-822d-2f7b1bc92f5f None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:03.033361 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-9944c796-8729-4310-922b-8a8c2bf89942 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:03.033361 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:03.033361 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:03.033361 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:03.033361 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:03.033361 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:03.033361 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:03.033361 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:03.033361 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:03.033361 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:03.033361 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:03.033361 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:03.033361 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:03.033361 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:03.033361 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:03.034330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:03.034330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:03.034330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:03.034330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:03.034330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:03.034330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:03.034330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:03.034330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:03.034330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:03.034330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:03.034330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:03.034330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:03.034330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:03.034330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:03.034330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:03.034330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:03.034330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:03.035171 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:03.035171 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:03.035171 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:03.035171 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:03.035171 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 257/513] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:03.035838 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9944c796-8729-4310-922b-8a8c2bf89942 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:03.047646 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-07983622-3a0e-42f3-822d-2f7b1bc92f5f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 21:50:03.047646 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:03.047646 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:03.047646 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:03.047646 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:03.047646 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:03.047646 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:03.047646 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:03.047646 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:03.047646 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:03.047646 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:03.047646 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:03.047646 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 21:50:03.047646 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 21:50:03.047646 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 21:50:03.047646 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 21:50:03.048552 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:03.048552 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:03.048552 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 21:50:03.048552 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:03.048552 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 21:50:03.048552 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:03.048552 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:03.048552 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:03.048552 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:03.048552 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:03.048552 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:03.048552 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:03.048552 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:03.048552 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:03.048552 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:03.048552 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:03.048552 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:03.048552 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:03.049937 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:03.049937 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:03.049937 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 21:50:03.049937 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 21:50:03.049937 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 21:50:03.049937 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 21:50:03.049937 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 21:50:03.049937 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 21:50:03.049937 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 21:50:03.049937 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:03.049937 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 257/514] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:50:02 2022] GET /identity/v3/users/cinder => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:03.050499 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07983622-3a0e-42f3-822d-2f7b1bc92f5f None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:03.100708 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:03.139439 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9944c796-8729-4310-922b-8a8c2bf89942 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:03.147732 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07983622-3a0e-42f3-822d-2f7b1bc92f5f None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:03.171848 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ca598-befe-4810-9d57-a92e3924a4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:03.172500 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ca598-befe-4810-9d57-a92e3924a4d8 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:03.172781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ca598-befe-4810-9d57-a92e3924a4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:03.173052 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ca598-befe-4810-9d57-a92e3924a4d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:03.173769 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970ca598-befe-4810-9d57-a92e3924a4d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:03.174071 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970ca598-befe-4810-9d57-a92e3924a4d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:03.175967 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970ca598-befe-4810-9d57-a92e3924a4d8 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:03.181589 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1794402d-7614-487b-9103-4557fd681ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:03.182241 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794402d-7614-487b-9103-4557fd681ef0 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:03.182531 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794402d-7614-487b-9103-4557fd681ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:03.182808 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794402d-7614-487b-9103-4557fd681ef0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:03.187375 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 258/515] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:03.188222 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ca598-befe-4810-9d57-a92e3924a4d8 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:03.194869 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:03.252185 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1794402d-7614-487b-9103-4557fd681ef0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:03.254173 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1794402d-7614-487b-9103-4557fd681ef0 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:03.309084 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ca598-befe-4810-9d57-a92e3924a4d8 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:03.332326 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:03.344085 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd57a524-dc0b-4036-b234-69c43811aeca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:03.344882 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd57a524-dc0b-4036-b234-69c43811aeca None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:03.345167 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd57a524-dc0b-4036-b234-69c43811aeca None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:03.345443 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd57a524-dc0b-4036-b234-69c43811aeca None admin] PATH_INFO: `/v3/projects/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:03.352602 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd57a524-dc0b-4036-b234-69c43811aeca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:03.354122 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd57a524-dc0b-4036-b234-69c43811aeca None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:03.358118 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 258/516] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:50:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 21:50:03.358957 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1794402d-7614-487b-9103-4557fd681ef0 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:03.368393 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-bd57a524-dc0b-4036-b234-69c43811aeca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:03.368393 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:03.368393 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:03.368393 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:03.368393 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:03.368393 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:03.368393 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:03.368393 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:03.368393 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:03.368393 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:03.368393 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:03.368393 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:03.368393 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 21:50:03.368393 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 21:50:03.368393 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 21:50:03.369242 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 21:50:03.369242 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:03.369242 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:03.369242 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:03.369242 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:03.369242 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:03.369242 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:03.369242 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:03.369242 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:03.369242 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:03.369242 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:03.369242 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:03.369242 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:03.369242 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:03.369242 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:03.369242 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:03.369242 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:03.370113 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 21:50:03.370113 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 21:50:03.370113 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 21:50:03.370113 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 21:50:03.370113 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 21:50:03.370113 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 21:50:03.370113 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:03.370113 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:03.370113 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 259/517] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:50:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:03.370585 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd57a524-dc0b-4036-b234-69c43811aeca None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:03.403251 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:03.460682 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1794402d-7614-487b-9103-4557fd681ef0 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:03.471413 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd57a524-dc0b-4036-b234-69c43811aeca None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:03.495076 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5df2af-1a4b-466a-8388-6b71ce9ed528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:03.495765 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5df2af-1a4b-466a-8388-6b71ce9ed528 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:03.496062 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5df2af-1a4b-466a-8388-6b71ce9ed528 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:03.496332 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5df2af-1a4b-466a-8388-6b71ce9ed528 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:03.497055 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5df2af-1a4b-466a-8388-6b71ce9ed528 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:03.497364 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5df2af-1a4b-466a-8388-6b71ce9ed528 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:03.498733 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5df2af-1a4b-466a-8388-6b71ce9ed528 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:03.505791 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ba64e-ca08-4a7f-ab49-d9d56bc0a93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:03.506402 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ba64e-ca08-4a7f-ab49-d9d56bc0a93b None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:03.506680 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ba64e-ca08-4a7f-ab49-d9d56bc0a93b None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:03.506953 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ba64e-ca08-4a7f-ab49-d9d56bc0a93b None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:03.507839 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679ba64e-ca08-4a7f-ab49-d9d56bc0a93b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:03.508145 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679ba64e-ca08-4a7f-ab49-d9d56bc0a93b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:03.509560 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679ba64e-ca08-4a7f-ab49-d9d56bc0a93b None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:03.510721 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 259/518] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:03.511743 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5df2af-1a4b-466a-8388-6b71ce9ed528 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:03.529810 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 260/519] 10.222.0.13 () {56 vars in 1137 bytes} [Tue Jun 7 21:50:03 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:03.530936 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ba64e-ca08-4a7f-ab49-d9d56bc0a93b None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:03.572692 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:03.619050 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5df2af-1a4b-466a-8388-6b71ce9ed528 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:03.620584 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:03.631697 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ba64e-ca08-4a7f-ab49-d9d56bc0a93b None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:03.653302 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3806c629-d55b-4707-8955-73603d5976c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:03.653975 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3806c629-d55b-4707-8955-73603d5976c2 None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:03.654257 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3806c629-d55b-4707-8955-73603d5976c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:03.654541 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3806c629-d55b-4707-8955-73603d5976c2 None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:03.655234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3806c629-d55b-4707-8955-73603d5976c2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:03.656685 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3806c629-d55b-4707-8955-73603d5976c2 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:03.665084 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33afcbe3-d63c-4346-9597-4013acdbc4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:03.665878 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33afcbe3-d63c-4346-9597-4013acdbc4a5 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:03.666168 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33afcbe3-d63c-4346-9597-4013acdbc4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:03.666446 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33afcbe3-d63c-4346-9597-4013acdbc4a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:03.676707 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 260/520] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Jun 7 21:50:03 2022] POST /identity/v3/users => generated 247 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 21:50:03.677534 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3806c629-d55b-4707-8955-73603d5976c2 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:03.678066 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33afcbe3-d63c-4346-9597-4013acdbc4a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:03.679578 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33afcbe3-d63c-4346-9597-4013acdbc4a5 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:03.699805 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-33afcbe3-d63c-4346-9597-4013acdbc4a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:03.699805 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:03.699805 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:03.699805 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:03.699805 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:03.699805 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:03.699805 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:03.699805 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:03.699805 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:03.699805 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:03.699805 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:03.699805 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:03.699805 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:03.699805 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:03.699805 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:03.700744 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:03.700744 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:03.700744 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:03.700744 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:03.700744 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:03.700744 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:03.700744 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:03.700744 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:03.700744 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:03.700744 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:03.700744 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:03.700744 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:03.700744 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:03.700744 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:03.700744 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:03.700744 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:03.700744 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:03.701593 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:03.701593 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:03.701593 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:03.701593 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:03.701593 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 261/521] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:03.701874 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33afcbe3-d63c-4346-9597-4013acdbc4a5 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:03.818202 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3806c629-d55b-4707-8955-73603d5976c2 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:03.819115 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:03.829537 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33afcbe3-d63c-4346-9597-4013acdbc4a5 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:03.859203 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f5267-e3b2-40dd-a1af-72f1eeef186d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:03.859940 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f5267-e3b2-40dd-a1af-72f1eeef186d None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:03.860229 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f5267-e3b2-40dd-a1af-72f1eeef186d None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:03.860513 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f5267-e3b2-40dd-a1af-72f1eeef186d None admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:03.861322 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96f5267-e3b2-40dd-a1af-72f1eeef186d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:03.861872 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96f5267-e3b2-40dd-a1af-72f1eeef186d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:03.863724 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96f5267-e3b2-40dd-a1af-72f1eeef186d None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:03.867394 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98257674-371e-4c6b-888c-2432ad74e13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:03.868207 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98257674-371e-4c6b-888c-2432ad74e13a None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:03.868500 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98257674-371e-4c6b-888c-2432ad74e13a None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:03.868779 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98257674-371e-4c6b-888c-2432ad74e13a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:03.876846 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98257674-371e-4c6b-888c-2432ad74e13a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:03.878952 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98257674-371e-4c6b-888c-2432ad74e13a None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:03.885024 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 261/522] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jun 7 21:50:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:03.885878 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f5267-e3b2-40dd-a1af-72f1eeef186d None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:03.898740 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-98257674-371e-4c6b-888c-2432ad74e13a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 21:50:03.898740 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:03.898740 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:03.898740 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:03.898740 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:03.898740 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:03.898740 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:03.898740 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:03.898740 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:03.898740 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:03.898740 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:03.898740 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:03.898740 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 21:50:03.898740 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 21:50:03.898740 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 21:50:03.898740 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 21:50:03.899704 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:03.899704 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:03.899704 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 21:50:03.899704 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:03.899704 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 21:50:03.899704 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:03.899704 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:03.899704 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:03.899704 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:03.899704 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:03.899704 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:03.899704 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:03.899704 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:03.899704 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:03.899704 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:03.899704 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:03.899704 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:03.899704 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:03.900614 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:03.900614 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:03.900614 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 21:50:03.900614 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 21:50:03.900614 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 21:50:03.900614 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 21:50:03.900614 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 21:50:03.900614 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 21:50:03.900614 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 21:50:03.900614 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:03.900614 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 262/523] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jun 7 21:50:03 2022] GET /identity/v3/users/neutron => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:03.901169 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98257674-371e-4c6b-888c-2432ad74e13a None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:03.908493 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:04.000283 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f5267-e3b2-40dd-a1af-72f1eeef186d None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:04.015684 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98257674-371e-4c6b-888c-2432ad74e13a None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:04.040566 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55969a4-9180-4d76-96a4-a99307aa3bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:04.041329 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55969a4-9180-4d76-96a4-a99307aa3bc4 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:04.041648 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55969a4-9180-4d76-96a4-a99307aa3bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:04.041927 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55969a4-9180-4d76-96a4-a99307aa3bc4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:04.052947 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d4cd5d-a4dd-43d6-a915-3580c468dbff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:04.053694 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d4cd5d-a4dd-43d6-a915-3580c468dbff None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:04.053980 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d4cd5d-a4dd-43d6-a915-3580c468dbff None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:04.054505 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d4cd5d-a4dd-43d6-a915-3580c468dbff None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:04.054972 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d4cd5d-a4dd-43d6-a915-3580c468dbff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:04.055277 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d4cd5d-a4dd-43d6-a915-3580c468dbff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:04.056693 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d4cd5d-a4dd-43d6-a915-3580c468dbff None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:04.076769 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 263/524] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:04.076769 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d4cd5d-a4dd-43d6-a915-3580c468dbff None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:04.092225 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:04.116359 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55969a4-9180-4d76-96a4-a99307aa3bc4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:04.117856 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55969a4-9180-4d76-96a4-a99307aa3bc4 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:04.137543 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:04.192652 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d4cd5d-a4dd-43d6-a915-3580c468dbff None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:04.230282 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4994bf-ed03-4e72-9dcb-cb8d60d7a40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:04.230931 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4994bf-ed03-4e72-9dcb-cb8d60d7a40b None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:04.231211 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4994bf-ed03-4e72-9dcb-cb8d60d7a40b None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:04.235974 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4994bf-ed03-4e72-9dcb-cb8d60d7a40b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:04.236122 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 262/525] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:50:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 21:50:04.236543 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4994bf-ed03-4e72-9dcb-cb8d60d7a40b None admin] RBAC: Adding query filter params (role.id=2c7f73c184d0406bbf2899465953342b, scope.project.id=0219dbda54274fdc9bdccbd960c30c6b, user.id=dbe0303ea0764347bfb054870ebba41f) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:04.236621 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55969a4-9180-4d76-96a4-a99307aa3bc4 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:04.236689 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4994bf-ed03-4e72-9dcb-cb8d60d7a40b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:04.236689 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4994bf-ed03-4e72-9dcb-cb8d60d7a40b None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:04.241237 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 264/526] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:50:04 2022] GET /identity/v3/role_assignments?user.id=dbe0303ea0764347bfb054870ebba41f&scope.project.id=0219dbda54274fdc9bdccbd960c30c6b&role.id=2c7f73c184d0406bbf2899465953342b => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:04.242528 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4994bf-ed03-4e72-9dcb-cb8d60d7a40b None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:04.349602 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55969a4-9180-4d76-96a4-a99307aa3bc4 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:04.357149 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:04.372083 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:04.388889 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4994bf-ed03-4e72-9dcb-cb8d60d7a40b None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:04.391490 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0add56f-3613-4077-bd70-18ab6ffed2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:04.392204 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0add56f-3613-4077-bd70-18ab6ffed2c5 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:04.392491 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0add56f-3613-4077-bd70-18ab6ffed2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:04.392769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0add56f-3613-4077-bd70-18ab6ffed2c5 None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:04.394596 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0add56f-3613-4077-bd70-18ab6ffed2c5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:04.394596 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0add56f-3613-4077-bd70-18ab6ffed2c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:04.396037 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0add56f-3613-4077-bd70-18ab6ffed2c5 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:04.421055 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 263/527] 10.222.0.13 () {56 vars in 1139 bytes} [Tue Jun 7 21:50:04 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:04.422128 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0add56f-3613-4077-bd70-18ab6ffed2c5 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:04.432929 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b017f874-8468-4ea8-823f-1e9b4418d66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:04.434133 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b017f874-8468-4ea8-823f-1e9b4418d66c None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:04.434709 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b017f874-8468-4ea8-823f-1e9b4418d66c None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:04.435189 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b017f874-8468-4ea8-823f-1e9b4418d66c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:04.444047 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b017f874-8468-4ea8-823f-1e9b4418d66c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:04.445904 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b017f874-8468-4ea8-823f-1e9b4418d66c None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:04.462010 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-b017f874-8468-4ea8-823f-1e9b4418d66c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:04.462010 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:04.462010 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:04.462010 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:04.462010 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:04.462010 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:04.462010 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:04.462010 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:04.462010 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:04.462010 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:04.462010 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:04.462010 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:04.462010 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 21:50:04.462010 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 21:50:04.462010 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 21:50:04.462906 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 21:50:04.462906 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:04.462906 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:04.462906 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:04.462906 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:04.462906 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:04.462906 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:04.462906 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:04.462906 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:04.462906 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:04.462906 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:04.462906 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:04.462906 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:04.462906 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:04.462906 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:04.462906 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:04.462906 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:04.463810 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 21:50:04.463810 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 21:50:04.463810 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 21:50:04.463810 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 21:50:04.463810 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 21:50:04.463810 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 21:50:04.463810 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:04.463810 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:04.463810 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 265/528] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:50:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:04.464891 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b017f874-8468-4ea8-823f-1e9b4418d66c None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:04.540055 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0add56f-3613-4077-bd70-18ab6ffed2c5 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:04.577506 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b017f874-8468-4ea8-823f-1e9b4418d66c None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:04.583944 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5588085-e204-4576-8e28-fb28d5d985f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:04.584756 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5588085-e204-4576-8e28-fb28d5d985f0 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:04.585027 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5588085-e204-4576-8e28-fb28d5d985f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:04.585302 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5588085-e204-4576-8e28-fb28d5d985f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:04.600220 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5588085-e204-4576-8e28-fb28d5d985f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:04.602126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5588085-e204-4576-8e28-fb28d5d985f0 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:04.615831 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a990bde-d6de-4fd1-83b2-5e25561428b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:04.616623 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a990bde-d6de-4fd1-83b2-5e25561428b4 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:04.617023 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a990bde-d6de-4fd1-83b2-5e25561428b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:04.617405 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a990bde-d6de-4fd1-83b2-5e25561428b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:04.622201 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:04.625353 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-c5588085-e204-4576-8e28-fb28d5d985f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:04.625353 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:04.625353 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:04.625353 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:04.625353 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:04.625353 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:04.625353 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:04.625353 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:04.625353 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:04.625353 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:04.625353 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:04.625353 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:04.625353 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:04.625353 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:04.625353 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:04.626385 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:04.626385 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:04.626385 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:04.626385 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:04.626385 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:04.626385 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:04.626385 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:04.626385 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:04.626385 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:04.626385 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:04.626385 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:04.626385 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:04.626385 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:04.626385 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:04.626385 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:04.626385 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:04.626385 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:04.627242 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:04.627242 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:04.627242 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:04.627242 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:04.627242 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 264/529] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:04.631399 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5588085-e204-4576-8e28-fb28d5d985f0 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:04.671973 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:04.696226 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a990bde-d6de-4fd1-83b2-5e25561428b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:04.698013 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a990bde-d6de-4fd1-83b2-5e25561428b4 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:04.770768 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5588085-e204-4576-8e28-fb28d5d985f0 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:04.805466 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 266/530] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:50:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 21:50:04.807198 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a990bde-d6de-4fd1-83b2-5e25561428b4 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:04.812652 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b903c8-1f7f-4a44-9d97-6678f18878e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:04.813312 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b903c8-1f7f-4a44-9d97-6678f18878e2 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:04.813593 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b903c8-1f7f-4a44-9d97-6678f18878e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:04.813864 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b903c8-1f7f-4a44-9d97-6678f18878e2 None admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:04.814650 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b903c8-1f7f-4a44-9d97-6678f18878e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:04.817134 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b903c8-1f7f-4a44-9d97-6678f18878e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:04.817326 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b903c8-1f7f-4a44-9d97-6678f18878e2 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:04.835817 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 265/531] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jun 7 21:50:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:04.837680 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b903c8-1f7f-4a44-9d97-6678f18878e2 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:04.858477 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:04.945107 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a990bde-d6de-4fd1-83b2-5e25561428b4 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:04.958633 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:04.977493 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b903c8-1f7f-4a44-9d97-6678f18878e2 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:04.986447 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b77a5-409f-48a2-82cc-6ca8a437ae44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:04.987192 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b77a5-409f-48a2-82cc-6ca8a437ae44 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:04.988261 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b77a5-409f-48a2-82cc-6ca8a437ae44 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:04.988546 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b77a5-409f-48a2-82cc-6ca8a437ae44 None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:04.989283 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1b77a5-409f-48a2-82cc-6ca8a437ae44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:04.989592 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1b77a5-409f-48a2-82cc-6ca8a437ae44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:04.991448 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1b77a5-409f-48a2-82cc-6ca8a437ae44 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:05.006532 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 267/532] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:05.007423 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b77a5-409f-48a2-82cc-6ca8a437ae44 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:05.022277 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0517d4-66d0-412c-976c-1f6dcd65715e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:05.024063 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0517d4-66d0-412c-976c-1f6dcd65715e None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:05.024506 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0517d4-66d0-412c-976c-1f6dcd65715e None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:05.024934 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0517d4-66d0-412c-976c-1f6dcd65715e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:05.040944 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0517d4-66d0-412c-976c-1f6dcd65715e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:05.043407 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0517d4-66d0-412c-976c-1f6dcd65715e None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:05.067900 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-ea0517d4-66d0-412c-976c-1f6dcd65715e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:05.067900 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:05.067900 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:05.067900 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:05.067900 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:05.067900 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:05.067900 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:05.067900 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:05.067900 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:05.067900 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:05.067900 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:05.067900 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:05.067900 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:05.067900 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:05.067900 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:05.069022 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:05.069022 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:05.069022 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:05.069022 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:05.069022 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:05.069022 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:05.069022 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:05.069022 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:05.069022 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:05.069022 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:05.069022 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:05.069022 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:05.069022 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:05.069022 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:05.069022 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:05.069022 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:05.069022 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:05.070091 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:05.070091 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:05.070091 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:05.070091 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:05.075995 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 266/533] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:05.076244 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0517d4-66d0-412c-976c-1f6dcd65715e None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:05.128974 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:05.136311 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b77a5-409f-48a2-82cc-6ca8a437ae44 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:05.180232 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f199ab87-5a9a-4ef1-b330-6d52d968ecdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:05.181055 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199ab87-5a9a-4ef1-b330-6d52d968ecdd None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:05.181491 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199ab87-5a9a-4ef1-b330-6d52d968ecdd None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:05.181891 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199ab87-5a9a-4ef1-b330-6d52d968ecdd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:05.183268 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f199ab87-5a9a-4ef1-b330-6d52d968ecdd None admin] RBAC: Adding query filter params (role.id=2c7f73c184d0406bbf2899465953342b, scope.project.id=0219dbda54274fdc9bdccbd960c30c6b, user.id=96987b04471944bc86ab44ff9936c732) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:05.184108 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f199ab87-5a9a-4ef1-b330-6d52d968ecdd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:05.185909 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f199ab87-5a9a-4ef1-b330-6d52d968ecdd None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:05.203361 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 268/534] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:50:05 2022] GET /identity/v3/role_assignments?user.id=96987b04471944bc86ab44ff9936c732&scope.project.id=0219dbda54274fdc9bdccbd960c30c6b&role.id=2c7f73c184d0406bbf2899465953342b => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:05.203361 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f199ab87-5a9a-4ef1-b330-6d52d968ecdd None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:05.224472 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0517d4-66d0-412c-976c-1f6dcd65715e None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:05.302281 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db91726f-f288-4fe2-87cb-6252a3f0464d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:05.303138 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91726f-f288-4fe2-87cb-6252a3f0464d None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:05.303444 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91726f-f288-4fe2-87cb-6252a3f0464d None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:05.304863 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91726f-f288-4fe2-87cb-6252a3f0464d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:05.320198 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db91726f-f288-4fe2-87cb-6252a3f0464d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:05.321713 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db91726f-f288-4fe2-87cb-6252a3f0464d None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:05.326903 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:05.359956 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-db91726f-f288-4fe2-87cb-6252a3f0464d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:05.359956 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:05.359956 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:05.359956 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:05.359956 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:05.359956 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:05.359956 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:05.359956 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:05.359956 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:05.359956 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:05.359956 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:05.359956 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:05.359956 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 21:50:05.359956 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 21:50:05.359956 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 21:50:05.360905 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 21:50:05.360905 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:05.360905 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:05.360905 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:05.360905 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:05.360905 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:05.360905 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:05.360905 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:05.360905 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:05.360905 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:05.360905 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:05.360905 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:05.360905 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:05.360905 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:05.360905 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:05.360905 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:05.360905 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:05.361769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 21:50:05.361769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 21:50:05.361769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 21:50:05.361769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 21:50:05.361769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 21:50:05.361769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 21:50:05.361769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:05.361769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:05.363891 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 267/535] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:50:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:05.366841 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db91726f-f288-4fe2-87cb-6252a3f0464d None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:05.400933 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f199ab87-5a9a-4ef1-b330-6d52d968ecdd None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:05.442016 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c6ca3-8704-460d-856b-b0bd4d7338ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:05.443968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c6ca3-8704-460d-856b-b0bd4d7338ac None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:05.443968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c6ca3-8704-460d-856b-b0bd4d7338ac None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:05.443968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c6ca3-8704-460d-856b-b0bd4d7338ac None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:05.444568 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c6ca3-8704-460d-856b-b0bd4d7338ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:05.444878 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c6ca3-8704-460d-856b-b0bd4d7338ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:05.447010 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c6ca3-8704-460d-856b-b0bd4d7338ac None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:05.466472 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 269/536] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:05.476510 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c6ca3-8704-460d-856b-b0bd4d7338ac None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:05.508476 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:05.593300 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db91726f-f288-4fe2-87cb-6252a3f0464d None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:05.603115 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c6ca3-8704-460d-856b-b0bd4d7338ac None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:05.643969 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45be360f-bfe8-4f64-a88f-5933fdd4a2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:05.644781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45be360f-bfe8-4f64-a88f-5933fdd4a2ef None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:05.645063 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45be360f-bfe8-4f64-a88f-5933fdd4a2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:05.645336 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45be360f-bfe8-4f64-a88f-5933fdd4a2ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:05.657952 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45be360f-bfe8-4f64-a88f-5933fdd4a2ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:05.660430 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45be360f-bfe8-4f64-a88f-5933fdd4a2ef None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:05.661456 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851a830-12f8-4ce0-a08f-9a57422773b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:05.662128 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851a830-12f8-4ce0-a08f-9a57422773b2 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:05.662411 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851a830-12f8-4ce0-a08f-9a57422773b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:05.662688 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851a830-12f8-4ce0-a08f-9a57422773b2 None admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:05.663523 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e851a830-12f8-4ce0-a08f-9a57422773b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:05.663836 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e851a830-12f8-4ce0-a08f-9a57422773b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:05.665271 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e851a830-12f8-4ce0-a08f-9a57422773b2 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:05.686908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-45be360f-bfe8-4f64-a88f-5933fdd4a2ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:05.686908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:05.686908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:05.686908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:05.686908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:05.686908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:05.686908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:05.686908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:05.686908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:05.686908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:05.686908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:05.686908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:05.686908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 21:50:05.686908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 21:50:05.686908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 21:50:05.687877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 21:50:05.687877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:05.687877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:05.687877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:05.687877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:05.687877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:05.687877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:05.687877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:05.687877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:05.687877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:05.687877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:05.687877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:05.687877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:05.687877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:05.687877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:05.687877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:05.687877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:05.688727 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 21:50:05.688727 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 21:50:05.688727 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 21:50:05.688727 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 21:50:05.688727 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 21:50:05.688727 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 21:50:05.688727 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:05.688727 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:05.691723 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 270/537] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:50:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:05.700408 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45be360f-bfe8-4f64-a88f-5933fdd4a2ef None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:05.702931 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 268/538] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jun 7 21:50:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:05.712454 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851a830-12f8-4ce0-a08f-9a57422773b2 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:05.823689 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:05.863753 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:05.901653 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45be360f-bfe8-4f64-a88f-5933fdd4a2ef None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:05.903803 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851a830-12f8-4ce0-a08f-9a57422773b2 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:05.948368 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d34d55f-15a1-4d3d-b3dd-bd5deeaa3cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:05.951955 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34d55f-15a1-4d3d-b3dd-bd5deeaa3cc5 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:05.952449 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34d55f-15a1-4d3d-b3dd-bd5deeaa3cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:05.952949 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34d55f-15a1-4d3d-b3dd-bd5deeaa3cc5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:05.953990 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d34d55f-15a1-4d3d-b3dd-bd5deeaa3cc5 None admin] RBAC: Adding query filter params (role.id=2c7f73c184d0406bbf2899465953342b, scope.project.id=0219dbda54274fdc9bdccbd960c30c6b, user.id=ea8a1e2cf1da4f87b8d4e16f525fc759) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:05.954821 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d34d55f-15a1-4d3d-b3dd-bd5deeaa3cc5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:05.958132 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d34d55f-15a1-4d3d-b3dd-bd5deeaa3cc5 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:05.969290 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 269/539] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:50:05 2022] GET /identity/v3/role_assignments?user.id=ea8a1e2cf1da4f87b8d4e16f525fc759&scope.project.id=0219dbda54274fdc9bdccbd960c30c6b&role.id=2c7f73c184d0406bbf2899465953342b => generated 259 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:05.985561 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab7e411-7249-4ed1-824d-7d3858ad2385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:05.987748 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab7e411-7249-4ed1-824d-7d3858ad2385 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:05.988401 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab7e411-7249-4ed1-824d-7d3858ad2385 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:05.988909 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab7e411-7249-4ed1-824d-7d3858ad2385 None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:05.989968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab7e411-7249-4ed1-824d-7d3858ad2385 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:05.990496 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab7e411-7249-4ed1-824d-7d3858ad2385 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:05.992304 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab7e411-7249-4ed1-824d-7d3858ad2385 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:06.021708 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 271/540] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jun 7 21:50:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:06.028063 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d34d55f-15a1-4d3d-b3dd-bd5deeaa3cc5 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:06.047358 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc36d1-2455-4bdc-9e9c-d2ba9d809db0 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:06.047358 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc36d1-2455-4bdc-9e9c-d2ba9d809db0 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:06.047358 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc36d1-2455-4bdc-9e9c-d2ba9d809db0 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:06.048492 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 272/541] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:06.054509 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98cca98-08b2-421e-95f2-0352123faa19 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:06.054815 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98cca98-08b2-421e-95f2-0352123faa19 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:06.055071 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98cca98-08b2-421e-95f2-0352123faa19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:06.087652 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-d98cca98-08b2-421e-95f2-0352123faa19 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:06.088994 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d98cca98-08b2-421e-95f2-0352123faa19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:06.171475 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 273/542] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:50:06.195057 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98cca98-08b2-421e-95f2-0352123faa19 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:06.226129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d34d55f-15a1-4d3d-b3dd-bd5deeaa3cc5 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:06.269425 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:06.287599 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221a94a-77ea-41d8-aa38-aaff45a5fbce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:06.288268 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221a94a-77ea-41d8-aa38-aaff45a5fbce None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:06.288551 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221a94a-77ea-41d8-aa38-aaff45a5fbce None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:06.288828 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221a94a-77ea-41d8-aa38-aaff45a5fbce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:06.289668 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8221a94a-77ea-41d8-aa38-aaff45a5fbce None admin] RBAC: Adding query filter params (role.id=2c7f73c184d0406bbf2899465953342b, scope.project.id=0219dbda54274fdc9bdccbd960c30c6b, user.id=10242251e6284217bfb79c67b1567f75) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:06.289983 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8221a94a-77ea-41d8-aa38-aaff45a5fbce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:06.291449 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8221a94a-77ea-41d8-aa38-aaff45a5fbce None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:06.303722 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:06.312580 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 270/543] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:50:06 2022] GET /identity/v3/role_assignments?user.id=10242251e6284217bfb79c67b1567f75&scope.project.id=0219dbda54274fdc9bdccbd960c30c6b&role.id=2c7f73c184d0406bbf2899465953342b => generated 259 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:06.348781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98cca98-08b2-421e-95f2-0352123faa19 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:06.388672 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223755cf-5434-4935-b055-7d741fb0d4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:06.389495 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223755cf-5434-4935-b055-7d741fb0d4ba None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:06.389780 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223755cf-5434-4935-b055-7d741fb0d4ba None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:06.390055 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223755cf-5434-4935-b055-7d741fb0d4ba None admin] PATH_INFO: `/v3/roles/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:06.403180 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223755cf-5434-4935-b055-7d741fb0d4ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:50:06.412006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-223755cf-5434-4935-b055-7d741fb0d4ba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 21:50:06.412006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:06.412006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:06.412006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:06.412006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:06.412006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:06.412006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:06.412006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:06.412006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:06.412006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:06.412006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:06.412006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:06.412006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:50:06.412006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:50:06.412006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:50:06.412006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:50:06.412997 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:50:06.412997 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:50:06.412997 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:50:06.412997 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:50:06.412997 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:06.412997 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:06.412997 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:06.412997 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:06.412997 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:06.412997 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:06.412997 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:06.412997 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:06.412997 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:06.412997 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:06.412997 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:06.412997 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:06.414482 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:06.414482 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:06.414482 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:06.414482 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:06.414482 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:50:06.414482 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:50:06.414482 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:50:06.414482 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:50:06.414482 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:50:06.414482 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:50:06.414482 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 21:50:06.414482 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:06.414482 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 274/544] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jun 7 21:50:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:50:06.419303 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221a94a-77ea-41d8-aa38-aaff45a5fbce None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:06.531098 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221a94a-77ea-41d8-aa38-aaff45a5fbce None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:06.573769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e7c990-e830-4e00-9d92-d78b648b5395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:06.574450 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7c990-e830-4e00-9d92-d78b648b5395 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:06.574739 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7c990-e830-4e00-9d92-d78b648b5395 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:06.575020 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7c990-e830-4e00-9d92-d78b648b5395 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:06.575866 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e7c990-e830-4e00-9d92-d78b648b5395 None admin] RBAC: Adding query filter params (name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:06.576179 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e7c990-e830-4e00-9d92-d78b648b5395 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:06.577607 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e7c990-e830-4e00-9d92-d78b648b5395 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:06.587311 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 271/545] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Jun 7 21:50:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:50:06.592760 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223755cf-5434-4935-b055-7d741fb0d4ba None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:06.751811 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223755cf-5434-4935-b055-7d741fb0d4ba None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:06.782327 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:06.788709 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a06e1eb-e45e-4c6b-8f74-5ade3698d64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:06.789513 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06e1eb-e45e-4c6b-8f74-5ade3698d64b None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:06.789795 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06e1eb-e45e-4c6b-8f74-5ade3698d64b None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:06.790065 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06e1eb-e45e-4c6b-8f74-5ade3698d64b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:06.801932 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a06e1eb-e45e-4c6b-8f74-5ade3698d64b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:06.803422 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a06e1eb-e45e-4c6b-8f74-5ade3698d64b None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:06.821817 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-7a06e1eb-e45e-4c6b-8f74-5ade3698d64b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:06.821817 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:06.821817 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:06.821817 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:06.821817 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:06.821817 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:06.821817 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:06.821817 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:06.821817 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:06.821817 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:06.821817 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:06.821817 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:06.821817 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:06.821817 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:06.821817 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:06.822711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:06.822711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:06.822711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:06.822711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:06.822711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:06.822711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:06.822711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:06.822711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:06.822711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:06.822711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:06.822711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:06.822711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:06.822711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:06.822711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:06.822711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:06.822711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:06.822711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:06.823613 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:06.823613 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:06.823613 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:06.823613 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:06.823613 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 275/546] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:06.832438 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e7c990-e830-4e00-9d92-d78b648b5395 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:06.890720 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:06.925924 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf0f4de-81fb-4622-940c-a3392fc11c27 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:06.926260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf0f4de-81fb-4622-940c-a3392fc11c27 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:06.926491 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf0f4de-81fb-4622-940c-a3392fc11c27 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:06.935817 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 276/547] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:06 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:06.944283 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e7c990-e830-4e00-9d92-d78b648b5395 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:06.944856 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36897c5d-3177-4b7b-822c-74e66e3a04e0 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:06.945149 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36897c5d-3177-4b7b-822c-74e66e3a04e0 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:06.945427 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36897c5d-3177-4b7b-822c-74e66e3a04e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:06.981711 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef063e-6d40-4102-badb-b30b6831450a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:06.982384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef063e-6d40-4102-badb-b30b6831450a None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:06.982675 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef063e-6d40-4102-badb-b30b6831450a None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:06.982961 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef063e-6d40-4102-badb-b30b6831450a None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:06.983858 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ef063e-6d40-4102-badb-b30b6831450a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:06.984181 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ef063e-6d40-4102-badb-b30b6831450a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:06.985749 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ef063e-6d40-4102-badb-b30b6831450a None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:06.991212 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-36897c5d-3177-4b7b-822c-74e66e3a04e0 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:06.992441 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-36897c5d-3177-4b7b-822c-74e66e3a04e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:07.004422 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 272/548] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:07.017342 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef063e-6d40-4102-badb-b30b6831450a None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:07.123011 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 277/549] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:50:07.146069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36897c5d-3177-4b7b-822c-74e66e3a04e0 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:07.250173 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36897c5d-3177-4b7b-822c-74e66e3a04e0 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:07.267739 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:07.269628 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef063e-6d40-4102-badb-b30b6831450a None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:07.287951 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:07.293067 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc04fc25-ccdc-44ec-a3d0-e35c177d55fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:07.293897 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc04fc25-ccdc-44ec-a3d0-e35c177d55fc None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:07.294152 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc04fc25-ccdc-44ec-a3d0-e35c177d55fc None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:07.294431 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc04fc25-ccdc-44ec-a3d0-e35c177d55fc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:07.314829 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc04fc25-ccdc-44ec-a3d0-e35c177d55fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:50:07.323232 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-bc04fc25-ccdc-44ec-a3d0-e35c177d55fc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 21:50:07.323232 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:07.323232 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:07.323232 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:07.323232 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:07.323232 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:07.323232 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:07.323232 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:07.323232 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:07.323232 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:07.323232 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:07.323232 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:07.323232 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:50:07.323232 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:50:07.323232 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:50:07.323232 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:50:07.324260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:50:07.324260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:50:07.324260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:50:07.324260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:50:07.324260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:07.324260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:07.324260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:07.324260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:07.324260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:07.324260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:07.324260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:07.324260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:07.324260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:07.324260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:07.324260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:07.324260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:07.325230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:07.325230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:07.325230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:07.325230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:07.325230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:50:07.325230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:50:07.325230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:50:07.325230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:50:07.325230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:50:07.325230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:50:07.325230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 21:50:07.325230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:07.335520 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aa50af-ddca-409a-9b0c-cc7fb5987300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:07.336341 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 278/550] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jun 7 21:50:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:50:07.336853 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aa50af-ddca-409a-9b0c-cc7fb5987300 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:07.337144 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aa50af-ddca-409a-9b0c-cc7fb5987300 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:07.337427 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aa50af-ddca-409a-9b0c-cc7fb5987300 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:07.343477 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc04fc25-ccdc-44ec-a3d0-e35c177d55fc None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:07.362917 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aa50af-ddca-409a-9b0c-cc7fb5987300 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:07.364427 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aa50af-ddca-409a-9b0c-cc7fb5987300 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:07.390829 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-96aa50af-ddca-409a-9b0c-cc7fb5987300 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 21:50:07.390829 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:07.390829 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:07.390829 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:07.390829 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:07.390829 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:07.390829 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:07.390829 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:07.390829 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:07.390829 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:07.390829 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:07.390829 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:07.390829 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 21:50:07.390829 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 21:50:07.390829 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 21:50:07.390829 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 21:50:07.392667 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:07.392667 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:07.392667 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 21:50:07.392667 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:07.392667 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 21:50:07.392667 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:07.392667 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:07.392667 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:07.392667 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:07.392667 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:07.392667 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:07.392667 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:07.392667 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:07.392667 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:07.392667 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:07.392667 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:07.392667 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:07.392667 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:07.395666 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:07.395666 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:07.395666 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 21:50:07.395666 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 21:50:07.395666 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 21:50:07.395666 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 21:50:07.395666 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 21:50:07.395666 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 21:50:07.395666 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 21:50:07.395666 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:07.395666 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 273/551] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jun 7 21:50:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:07.399987 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aa50af-ddca-409a-9b0c-cc7fb5987300 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:07.474130 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc04fc25-ccdc-44ec-a3d0-e35c177d55fc None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:07.509472 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80a0ee-2463-4165-92a1-013e8ebbb21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:07.510141 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80a0ee-2463-4165-92a1-013e8ebbb21d None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:07.510427 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80a0ee-2463-4165-92a1-013e8ebbb21d None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:07.510705 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80a0ee-2463-4165-92a1-013e8ebbb21d None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:07.511491 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d80a0ee-2463-4165-92a1-013e8ebbb21d None admin] RBAC: Adding query filter params (name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:07.511800 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d80a0ee-2463-4165-92a1-013e8ebbb21d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:07.513184 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d80a0ee-2463-4165-92a1-013e8ebbb21d None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:07.522288 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 279/552] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Jun 7 21:50:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:07.527684 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80a0ee-2463-4165-92a1-013e8ebbb21d None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:07.570926 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:07.631894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80a0ee-2463-4165-92a1-013e8ebbb21d None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:07.668513 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053c0970-949c-4a4a-a528-57d8e7bd2b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:07.669233 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053c0970-949c-4a4a-a528-57d8e7bd2b76 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:07.669518 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053c0970-949c-4a4a-a528-57d8e7bd2b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:07.669792 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053c0970-949c-4a4a-a528-57d8e7bd2b76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:07.670056 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aa50af-ddca-409a-9b0c-cc7fb5987300 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:07.682453 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053c0970-949c-4a4a-a528-57d8e7bd2b76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:07.683946 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053c0970-949c-4a4a-a528-57d8e7bd2b76 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:07.702543 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-053c0970-949c-4a4a-a528-57d8e7bd2b76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:07.702543 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:07.702543 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:07.702543 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:07.702543 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:07.702543 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:07.702543 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:07.702543 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:07.702543 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:07.702543 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:07.702543 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:07.702543 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:07.702543 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:07.702543 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:07.702543 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:07.703489 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:07.703489 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:07.703489 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:07.703489 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:07.703489 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:07.703489 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:07.703489 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:07.703489 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:07.703489 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:07.703489 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:07.703489 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:07.703489 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:07.703489 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:07.703489 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:07.703489 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:07.703489 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:07.703489 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:07.704467 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:07.704467 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:07.704467 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:07.704467 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:07.704467 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 280/553] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:07.710736 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053c0970-949c-4a4a-a528-57d8e7bd2b76 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:07.808571 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994dfb21-d74f-4f96-9227-a6b9b4f7fa03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:07.809712 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994dfb21-d74f-4f96-9227-a6b9b4f7fa03 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:07.810212 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994dfb21-d74f-4f96-9227-a6b9b4f7fa03 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:07.810706 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994dfb21-d74f-4f96-9227-a6b9b4f7fa03 None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:07.811808 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994dfb21-d74f-4f96-9227-a6b9b4f7fa03 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:07.812363 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994dfb21-d74f-4f96-9227-a6b9b4f7fa03 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:07.814000 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994dfb21-d74f-4f96-9227-a6b9b4f7fa03 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:07.819431 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:07.832626 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053c0970-949c-4a4a-a528-57d8e7bd2b76 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:07.854779 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 274/554] 10.222.0.13 () {56 vars in 1135 bytes} [Tue Jun 7 21:50:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 456 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:07.860149 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994dfb21-d74f-4f96-9227-a6b9b4f7fa03 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:07.874229 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e5f10-c8dd-4989-b7b3-8d551d4f7b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:07.874947 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e5f10-c8dd-4989-b7b3-8d551d4f7b0b None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:07.875238 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e5f10-c8dd-4989-b7b3-8d551d4f7b0b None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:07.875541 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e5f10-c8dd-4989-b7b3-8d551d4f7b0b None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:07.876271 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6e5f10-c8dd-4989-b7b3-8d551d4f7b0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:07.876582 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6e5f10-c8dd-4989-b7b3-8d551d4f7b0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:07.878672 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6e5f10-c8dd-4989-b7b3-8d551d4f7b0b None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:07.892208 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 281/555] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:50:07.898292 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e5f10-c8dd-4989-b7b3-8d551d4f7b0b None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:07.942452 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:08.012900 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e5f10-c8dd-4989-b7b3-8d551d4f7b0b None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:08.063262 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad0324a-0a46-4854-a19a-a140c6fcff63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:08.064158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0324a-0a46-4854-a19a-a140c6fcff63 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:08.064443 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0324a-0a46-4854-a19a-a140c6fcff63 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:08.064715 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0324a-0a46-4854-a19a-a140c6fcff63 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:08.083979 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad0324a-0a46-4854-a19a-a140c6fcff63 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:08.087124 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad0324a-0a46-4854-a19a-a140c6fcff63 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:08.114732 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-4ad0324a-0a46-4854-a19a-a140c6fcff63 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 21:50:08.114732 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:08.114732 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:08.114732 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:08.114732 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:08.114732 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:08.114732 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:08.114732 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:08.114732 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:08.114732 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:08.114732 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:08.114732 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:08.114732 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 21:50:08.114732 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 21:50:08.114732 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 21:50:08.114732 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 21:50:08.115908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:08.115908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:08.115908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 21:50:08.115908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:08.115908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 21:50:08.115908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:08.115908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:08.115908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:08.115908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:08.115908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:08.115908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:08.115908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:08.115908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:08.115908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:08.115908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:08.115908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:08.115908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:08.115908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:08.117325 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:08.117325 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:08.117325 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 21:50:08.117325 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 21:50:08.117325 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 21:50:08.117325 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 21:50:08.117325 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 21:50:08.117325 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 21:50:08.117325 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 21:50:08.117325 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:08.117882 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 282/556] 10.222.0.13 () {56 vars in 1093 bytes} [Tue Jun 7 21:50:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:50:08.125061 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad0324a-0a46-4854-a19a-a140c6fcff63 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:08.141098 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994dfb21-d74f-4f96-9227-a6b9b4f7fa03 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:08.147665 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:08.183115 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fdacba-6ce6-4c8f-8d08-f785366f6526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:08.184199 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fdacba-6ce6-4c8f-8d08-f785366f6526 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:08.184699 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fdacba-6ce6-4c8f-8d08-f785366f6526 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:08.185194 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fdacba-6ce6-4c8f-8d08-f785366f6526 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:08.252062 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad0324a-0a46-4854-a19a-a140c6fcff63 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:08.257644 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fdacba-6ce6-4c8f-8d08-f785366f6526 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:08.259363 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fdacba-6ce6-4c8f-8d08-f785366f6526 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:08.287660 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:08.294934 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd15de-f744-49be-8f8d-32815dc5e0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:08.295649 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd15de-f744-49be-8f8d-32815dc5e0d8 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:08.295936 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd15de-f744-49be-8f8d-32815dc5e0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:08.296237 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd15de-f744-49be-8f8d-32815dc5e0d8 None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:08.297980 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fd15de-f744-49be-8f8d-32815dc5e0d8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:08.298329 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fd15de-f744-49be-8f8d-32815dc5e0d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:08.300529 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fd15de-f744-49be-8f8d-32815dc5e0d8 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:08.323475 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 283/557] 10.222.0.13 () {56 vars in 1133 bytes} [Tue Jun 7 21:50:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:08.325148 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3432ab98-ebd0-495d-bfd0-ba9585bf7f9d None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:08.325649 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3432ab98-ebd0-495d-bfd0-ba9585bf7f9d None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:08.326140 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3432ab98-ebd0-495d-bfd0-ba9585bf7f9d None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:08.327700 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 284/558] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:08.333146 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3432ab98-ebd0-495d-bfd0-ba9585bf7f9d None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:08.388573 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 275/559] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:50:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 21:50:08.388573 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da2219-02ed-42ae-8d01-0fbab5e3a056 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:08.388573 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da2219-02ed-42ae-8d01-0fbab5e3a056 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:08.388573 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da2219-02ed-42ae-8d01-0fbab5e3a056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:08.406719 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-d3da2219-02ed-42ae-8d01-0fbab5e3a056 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:08.408136 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-d3da2219-02ed-42ae-8d01-0fbab5e3a056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:08.421904 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:08.452761 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3432ab98-ebd0-495d-bfd0-ba9585bf7f9d None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:08.488876 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 276/560] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:50:08.490108 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da2219-02ed-42ae-8d01-0fbab5e3a056 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:08.502407 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cbccd5-c8f5-4228-9048-6b41bbab25b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:08.504025 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cbccd5-c8f5-4228-9048-6b41bbab25b7 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:08.504508 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cbccd5-c8f5-4228-9048-6b41bbab25b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:08.504978 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cbccd5-c8f5-4228-9048-6b41bbab25b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:08.517258 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cbccd5-c8f5-4228-9048-6b41bbab25b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:08.519172 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cbccd5-c8f5-4228-9048-6b41bbab25b7 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:08.538721 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-14cbccd5-c8f5-4228-9048-6b41bbab25b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:08.538721 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:08.538721 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:08.538721 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:08.538721 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:08.538721 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:08.538721 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:08.538721 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:08.538721 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:08.538721 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:08.538721 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:08.538721 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:08.538721 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:08.538721 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:08.538721 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:08.539701 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:08.539701 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:08.539701 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:08.539701 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:08.539701 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:08.539701 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:08.539701 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:08.539701 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:08.539701 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:08.539701 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:08.539701 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:08.539701 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:08.539701 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:08.539701 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:08.539701 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:08.539701 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:08.539701 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:08.540601 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:08.540601 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:08.540601 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:08.540601 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:08.540601 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 285/561] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:08.541759 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15a7a8-edb5-4733-93af-6bafa39a2fd9 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:08.541759 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15a7a8-edb5-4733-93af-6bafa39a2fd9 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:08.541759 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15a7a8-edb5-4733-93af-6bafa39a2fd9 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:08.543231 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 286/562] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:08.543962 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b15a7a8-edb5-4733-93af-6bafa39a2fd9 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:08.567233 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:08.613550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da2219-02ed-42ae-8d01-0fbab5e3a056 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:08.649662 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b15a7a8-edb5-4733-93af-6bafa39a2fd9 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:08.650707 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e362aec-a0de-454e-92e9-dabcf7e8ed2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:08.651643 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e362aec-a0de-454e-92e9-dabcf7e8ed2f None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:08.652040 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e362aec-a0de-454e-92e9-dabcf7e8ed2f None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:08.652320 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e362aec-a0de-454e-92e9-dabcf7e8ed2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:08.662968 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e362aec-a0de-454e-92e9-dabcf7e8ed2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:08.664678 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e362aec-a0de-454e-92e9-dabcf7e8ed2f None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:08.667275 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:08.684589 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-0e362aec-a0de-454e-92e9-dabcf7e8ed2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:08.684589 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:08.684589 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:08.684589 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:08.684589 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:08.684589 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:08.684589 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:08.684589 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:08.684589 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:08.684589 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:08.684589 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:08.684589 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:08.684589 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:08.684589 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:08.684589 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:08.685531 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:08.685531 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:08.685531 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:08.685531 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:08.685531 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:08.685531 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:08.685531 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:08.685531 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:08.685531 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:08.685531 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:08.685531 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:08.685531 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:08.685531 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:08.685531 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:08.685531 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:08.685531 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:08.685531 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:08.686437 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:08.686437 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:08.686437 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:08.686437 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:08.686658 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca0c1d-6057-4db9-933b-052920885f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:08.686658 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca0c1d-6057-4db9-933b-052920885f87 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:08.687463 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca0c1d-6057-4db9-933b-052920885f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:08.687463 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca0c1d-6057-4db9-933b-052920885f87 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:08.691367 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 277/563] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:08.692461 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acefaeb6-531b-4c69-9e25-e64848c6fee8 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:08.692747 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acefaeb6-531b-4c69-9e25-e64848c6fee8 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:08.693023 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acefaeb6-531b-4c69-9e25-e64848c6fee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:08.702735 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bca0c1d-6057-4db9-933b-052920885f87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:50:08.710192 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-acefaeb6-531b-4c69-9e25-e64848c6fee8 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:08.710536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-7bca0c1d-6057-4db9-933b-052920885f87 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 21:50:08.710536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:08.710536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:08.710536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:08.710536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:08.710536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:08.710536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:08.710536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:08.710536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:08.710536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:08.710536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:08.710536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:08.710536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:50:08.710536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:50:08.710536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:50:08.710536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:50:08.711526 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:50:08.711526 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:50:08.711526 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:50:08.711526 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:50:08.711526 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:08.711526 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:08.711526 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:08.711526 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:08.711526 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:08.711526 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:08.711526 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:08.711526 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:08.711526 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:08.711526 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:08.711526 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:08.711526 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:08.717507 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:08.717507 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:08.717507 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:08.717507 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:08.717507 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:50:08.717507 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:50:08.717507 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:50:08.717507 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:50:08.717507 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:50:08.717507 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:50:08.717507 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 21:50:08.717507 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:08.718185 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-acefaeb6-531b-4c69-9e25-e64848c6fee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:08.718268 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 287/564] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jun 7 21:50:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:08.718268 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca0c1d-6057-4db9-933b-052920885f87 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:08.794050 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 278/565] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:50:08.795289 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acefaeb6-531b-4c69-9e25-e64848c6fee8 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:08.840466 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:08.844871 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca0c1d-6057-4db9-933b-052920885f87 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:08.882445 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb28855-3a7b-44cc-9ec3-610c59316477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:08.883209 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb28855-3a7b-44cc-9ec3-610c59316477 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:08.883530 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb28855-3a7b-44cc-9ec3-610c59316477 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:08.883802 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb28855-3a7b-44cc-9ec3-610c59316477 None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:08.884535 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb28855-3a7b-44cc-9ec3-610c59316477 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:08.884839 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb28855-3a7b-44cc-9ec3-610c59316477 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:08.886278 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb28855-3a7b-44cc-9ec3-610c59316477 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:08.899826 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 288/566] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:08.900675 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb28855-3a7b-44cc-9ec3-610c59316477 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:08.912662 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acefaeb6-531b-4c69-9e25-e64848c6fee8 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:08.935958 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:08.949308 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab8bd48-39f0-4b93-8751-76a76406eb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:08.949970 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8bd48-39f0-4b93-8751-76a76406eb93 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:08.950249 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8bd48-39f0-4b93-8751-76a76406eb93 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:08.950521 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8bd48-39f0-4b93-8751-76a76406eb93 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:08.951238 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab8bd48-39f0-4b93-8751-76a76406eb93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:08.951567 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab8bd48-39f0-4b93-8751-76a76406eb93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:08.952921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab8bd48-39f0-4b93-8751-76a76406eb93 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:08.964853 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 279/567] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:08.964853 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab8bd48-39f0-4b93-8751-76a76406eb93 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:09.015230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb28855-3a7b-44cc-9ec3-610c59316477 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:09.051659 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2027a500-813b-4750-ba42-5b3bc014b394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:09.052796 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027a500-813b-4750-ba42-5b3bc014b394 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:09.053340 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027a500-813b-4750-ba42-5b3bc014b394 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:09.053846 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027a500-813b-4750-ba42-5b3bc014b394 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:09.054772 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2027a500-813b-4750-ba42-5b3bc014b394 None admin] RBAC: Adding query filter params (name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:09.055287 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2027a500-813b-4750-ba42-5b3bc014b394 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:09.056939 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2027a500-813b-4750-ba42-5b3bc014b394 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:09.067064 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 289/568] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Jun 7 21:50:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:09.068332 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2027a500-813b-4750-ba42-5b3bc014b394 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:09.078456 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:09.081909 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab8bd48-39f0-4b93-8751-76a76406eb93 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:09.117029 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316555d0-0b72-44e0-a176-68f7925b25b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:09.117829 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316555d0-0b72-44e0-a176-68f7925b25b8 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:09.118112 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316555d0-0b72-44e0-a176-68f7925b25b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:09.118390 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316555d0-0b72-44e0-a176-68f7925b25b8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:09.129264 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316555d0-0b72-44e0-a176-68f7925b25b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:50:09.137342 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-316555d0-0b72-44e0-a176-68f7925b25b8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 21:50:09.137342 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:09.137342 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:09.137342 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:09.137342 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:09.137342 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:09.137342 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:09.137342 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:09.137342 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:09.137342 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:09.137342 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:09.137342 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:09.137342 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:50:09.137342 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:50:09.137342 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:50:09.137342 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:50:09.138330 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:50:09.138330 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:50:09.138330 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:50:09.138330 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:50:09.138330 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:09.138330 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:09.138330 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:09.138330 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:09.138330 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:09.138330 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:09.138330 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:09.138330 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:09.138330 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:09.138330 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:09.138330 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:09.138330 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:09.139161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:09.139161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:09.139161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:09.139161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:09.139161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:50:09.139161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:50:09.139161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:50:09.139161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:50:09.139161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:50:09.139161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:50:09.139161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 21:50:09.139161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:09.143613 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 280/569] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jun 7 21:50:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:09.144162 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316555d0-0b72-44e0-a176-68f7925b25b8 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:09.177938 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:09.182010 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2027a500-813b-4750-ba42-5b3bc014b394 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:09.218223 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7ffa22-c318-4457-8d86-4ac96108e83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:09.219012 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ffa22-c318-4457-8d86-4ac96108e83e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:09.219293 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ffa22-c318-4457-8d86-4ac96108e83e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:09.219596 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ffa22-c318-4457-8d86-4ac96108e83e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:09.226554 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7ffa22-c318-4457-8d86-4ac96108e83e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:09.228024 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7ffa22-c318-4457-8d86-4ac96108e83e None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:09.242357 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-aa7ffa22-c318-4457-8d86-4ac96108e83e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:09.242357 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:09.242357 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:09.242357 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:09.242357 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:09.242357 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:09.242357 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:09.242357 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:09.242357 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:09.242357 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:09.242357 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:09.242357 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:09.242357 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 21:50:09.242357 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 21:50:09.242357 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 21:50:09.243429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 21:50:09.243429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:09.243429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:09.243429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:09.243429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:09.243429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:09.243429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:09.243429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:09.243429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:09.243429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:09.243429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:09.243429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:09.243429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:09.243429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:09.243429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:09.243429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:09.243429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:09.244330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 21:50:09.244330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 21:50:09.244330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 21:50:09.244330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 21:50:09.244330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 21:50:09.244330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 21:50:09.244330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:09.244330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:09.244330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 290/570] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:50:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:09.244842 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7ffa22-c318-4457-8d86-4ac96108e83e None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:09.251939 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316555d0-0b72-44e0-a176-68f7925b25b8 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:09.286389 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c4d280-ff48-45f4-a40e-005108cc6eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:09.287174 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c4d280-ff48-45f4-a40e-005108cc6eec None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:09.287480 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c4d280-ff48-45f4-a40e-005108cc6eec None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:09.287759 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c4d280-ff48-45f4-a40e-005108cc6eec None admin] PATH_INFO: `/v3/projects/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:09.294245 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c4d280-ff48-45f4-a40e-005108cc6eec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:09.295693 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c4d280-ff48-45f4-a40e-005108cc6eec None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:09.310998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-a2c4d280-ff48-45f4-a40e-005108cc6eec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:09.310998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:09.310998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:09.310998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:09.310998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:09.310998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:09.310998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:09.310998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:09.310998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:09.310998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:09.310998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:09.310998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:09.310998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 21:50:09.310998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 21:50:09.310998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 21:50:09.312043 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 21:50:09.312043 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:09.312043 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:09.312043 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:09.312043 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:09.312043 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:09.312043 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:09.312043 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:09.312043 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:09.312043 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:09.312043 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:09.312043 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:09.312043 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:09.312043 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:09.312043 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:09.312043 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:09.312043 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:09.312973 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 21:50:09.312973 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 21:50:09.312973 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 21:50:09.312973 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 21:50:09.312973 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 21:50:09.312973 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 21:50:09.312973 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:09.312973 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:09.312973 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 281/571] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:50:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:09.313468 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c4d280-ff48-45f4-a40e-005108cc6eec None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:09.334143 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:09.348308 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7ffa22-c318-4457-8d86-4ac96108e83e None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:09.383949 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0226169-6aee-4888-949e-4fb17d828af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:09.384737 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0226169-6aee-4888-949e-4fb17d828af6 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:09.385017 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0226169-6aee-4888-949e-4fb17d828af6 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:09.385287 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0226169-6aee-4888-949e-4fb17d828af6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:09.396050 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0226169-6aee-4888-949e-4fb17d828af6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:09.397509 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0226169-6aee-4888-949e-4fb17d828af6 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:09.413829 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c4d280-ff48-45f4-a40e-005108cc6eec None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:09.415170 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-d0226169-6aee-4888-949e-4fb17d828af6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:09.415170 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:09.415170 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:09.415170 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:09.415170 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:09.415170 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:09.415170 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:09.415170 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:09.415170 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:09.415170 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:09.415170 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:09.415170 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:09.415170 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:09.415170 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:09.415170 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:09.416062 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:09.416062 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:09.416062 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:09.416062 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:09.416062 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:09.416062 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:09.416062 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:09.416062 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:09.416062 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:09.416062 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:09.416062 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:09.416062 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:09.416062 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:09.416062 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:09.416062 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:09.416062 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:09.416062 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:09.416961 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:09.416961 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:09.416961 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:09.416961 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:09.416961 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 291/572] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:09.417683 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2431da9-511d-40f2-8949-78f10a74ebe0 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:09.417808 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2431da9-511d-40f2-8949-78f10a74ebe0 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:09.418072 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2431da9-511d-40f2-8949-78f10a74ebe0 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:09.419300 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 292/573] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:09.419998 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2431da9-511d-40f2-8949-78f10a74ebe0 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:09.433597 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:09.454319 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad68a85-2e44-44b0-93f0-35c0902eb732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:09.454986 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad68a85-2e44-44b0-93f0-35c0902eb732 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:09.455293 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad68a85-2e44-44b0-93f0-35c0902eb732 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:09.455639 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad68a85-2e44-44b0-93f0-35c0902eb732 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:09.456501 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad68a85-2e44-44b0-93f0-35c0902eb732 None admin] RBAC: Adding query filter params (name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:09.456858 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad68a85-2e44-44b0-93f0-35c0902eb732 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:09.458302 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad68a85-2e44-44b0-93f0-35c0902eb732 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:09.466223 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 282/574] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Jun 7 21:50:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:09.466223 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad68a85-2e44-44b0-93f0-35c0902eb732 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:09.532269 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2431da9-511d-40f2-8949-78f10a74ebe0 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:09.548235 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:09.567079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad68a85-2e44-44b0-93f0-35c0902eb732 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:09.567230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acb204a-5f19-4ae9-ab06-7201d7456d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:09.567878 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb204a-5f19-4ae9-ab06-7201d7456d06 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:09.568156 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb204a-5f19-4ae9-ab06-7201d7456d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:09.568434 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb204a-5f19-4ae9-ab06-7201d7456d06 None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:09.569208 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acb204a-5f19-4ae9-ab06-7201d7456d06 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:09.569513 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acb204a-5f19-4ae9-ab06-7201d7456d06 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:09.570939 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acb204a-5f19-4ae9-ab06-7201d7456d06 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:09.582924 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 293/575] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jun 7 21:50:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:09.583737 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acb204a-5f19-4ae9-ab06-7201d7456d06 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:09.600823 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babb7f6f-4a63-49ad-b66e-0ffdc05e8ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:09.601484 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb7f6f-4a63-49ad-b66e-0ffdc05e8ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:09.601765 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb7f6f-4a63-49ad-b66e-0ffdc05e8ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:09.602039 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb7f6f-4a63-49ad-b66e-0ffdc05e8ba2 None admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:09.602825 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babb7f6f-4a63-49ad-b66e-0ffdc05e8ba2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:09.603128 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babb7f6f-4a63-49ad-b66e-0ffdc05e8ba2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:09.604567 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babb7f6f-4a63-49ad-b66e-0ffdc05e8ba2 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:09.617015 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 283/576] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jun 7 21:50:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:09.618225 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e34c3e-062b-4d17-b181-5c4c8b52216f None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:09.618512 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e34c3e-062b-4d17-b181-5c4c8b52216f None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:09.618791 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e34c3e-062b-4d17-b181-5c4c8b52216f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:09.634953 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-43e34c3e-062b-4d17-b181-5c4c8b52216f None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:09.636155 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-43e34c3e-062b-4d17-b181-5c4c8b52216f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:09.655286 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:09.686411 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acb204a-5f19-4ae9-ab06-7201d7456d06 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:09.701750 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 284/577] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:50:09.702411 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e34c3e-062b-4d17-b181-5c4c8b52216f None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:09.736468 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef37665-9dea-4556-be20-541c65e313c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:09.737848 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef37665-9dea-4556-be20-541c65e313c4 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:09.738136 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef37665-9dea-4556-be20-541c65e313c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:09.738413 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef37665-9dea-4556-be20-541c65e313c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:09.739135 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef37665-9dea-4556-be20-541c65e313c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:09.739483 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef37665-9dea-4556-be20-541c65e313c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:09.740874 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef37665-9dea-4556-be20-541c65e313c4 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:09.752524 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 294/578] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:09.753945 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef37665-9dea-4556-be20-541c65e313c4 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:09.777349 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:09.808076 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e34c3e-062b-4d17-b181-5c4c8b52216f None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:09.841410 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3d94b6-9ae7-4c27-bee6-dc07f9ce6e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:09.842190 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d94b6-9ae7-4c27-bee6-dc07f9ce6e9b None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:09.842470 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d94b6-9ae7-4c27-bee6-dc07f9ce6e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:09.842746 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d94b6-9ae7-4c27-bee6-dc07f9ce6e9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:09.852280 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef37665-9dea-4556-be20-541c65e313c4 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:09.853441 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3d94b6-9ae7-4c27-bee6-dc07f9ce6e9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:09.854873 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3d94b6-9ae7-4c27-bee6-dc07f9ce6e9b None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:09.872863 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-be3d94b6-9ae7-4c27-bee6-dc07f9ce6e9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:09.872863 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:09.872863 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:09.872863 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:09.872863 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:09.872863 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:09.872863 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:09.872863 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:09.872863 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:09.872863 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:09.872863 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:09.872863 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:09.872863 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:09.872863 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:09.872863 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:09.873862 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:09.873862 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:09.873862 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:09.873862 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:09.873862 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:09.873862 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:09.873862 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:09.873862 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:09.873862 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:09.873862 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:09.873862 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:09.873862 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:09.873862 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:09.873862 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:09.873862 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:09.873862 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:09.873862 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:09.875445 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:09.875445 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:09.875445 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:09.875445 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:09.875445 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 285/579] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:09.875445 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3d94b6-9ae7-4c27-bee6-dc07f9ce6e9b None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:09.885222 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:09.891551 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577dba81-f0c0-4025-b5f2-1ff06e555969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:09.892453 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577dba81-f0c0-4025-b5f2-1ff06e555969 None admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:09.892734 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577dba81-f0c0-4025-b5f2-1ff06e555969 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:09.893357 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577dba81-f0c0-4025-b5f2-1ff06e555969 None admin] PATH_INFO: `/v3/projects/0219dbda54274fdc9bdccbd960c30c6b/users/dbe0303ea0764347bfb054870ebba41f/roles/2c7f73c184d0406bbf2899465953342b` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:09.907547 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577dba81-f0c0-4025-b5f2-1ff06e555969 None admin] RBAC: Authorizing `identity:create_grant(project_id=0219dbda54274fdc9bdccbd960c30c6b, user_id=dbe0303ea0764347bfb054870ebba41f, role_id=2c7f73c184d0406bbf2899465953342b)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:09.909034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577dba81-f0c0-4025-b5f2-1ff06e555969 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:09.920727 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 295/580] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jun 7 21:50:09 2022] PUT /identity/v3/projects/0219dbda54274fdc9bdccbd960c30c6b/users/dbe0303ea0764347bfb054870ebba41f/roles/2c7f73c184d0406bbf2899465953342b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 21:50:09.921516 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577dba81-f0c0-4025-b5f2-1ff06e555969 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:09.990340 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3d94b6-9ae7-4c27-bee6-dc07f9ce6e9b None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:10.020792 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:10.034479 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef94d36c-263c-4a08-9497-3b1b9c2187a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:10.035250 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef94d36c-263c-4a08-9497-3b1b9c2187a4 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:10.035608 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef94d36c-263c-4a08-9497-3b1b9c2187a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:10.035892 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef94d36c-263c-4a08-9497-3b1b9c2187a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:10.036718 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef94d36c-263c-4a08-9497-3b1b9c2187a4 None admin] RBAC: Adding query filter params (role.id=2c7f73c184d0406bbf2899465953342b, scope.project.id=0219dbda54274fdc9bdccbd960c30c6b, user.id=7a1261563b6d4e50a4ffeea6a32bdb01) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:10.037028 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef94d36c-263c-4a08-9497-3b1b9c2187a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:10.038163 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577dba81-f0c0-4025-b5f2-1ff06e555969 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:10.038609 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef94d36c-263c-4a08-9497-3b1b9c2187a4 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:10.055412 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 286/581] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:50:09 2022] GET /identity/v3/role_assignments?user.id=7a1261563b6d4e50a4ffeea6a32bdb01&scope.project.id=0219dbda54274fdc9bdccbd960c30c6b&role.id=2c7f73c184d0406bbf2899465953342b => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:10.055412 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef94d36c-263c-4a08-9497-3b1b9c2187a4 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:10.091208 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33883d9-8561-4a7a-9f9e-d9a6c63564d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:10.092252 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33883d9-8561-4a7a-9f9e-d9a6c63564d0 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:10.092605 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33883d9-8561-4a7a-9f9e-d9a6c63564d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:10.092890 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33883d9-8561-4a7a-9f9e-d9a6c63564d0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:10.111209 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33883d9-8561-4a7a-9f9e-d9a6c63564d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:50:10.122037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-f33883d9-8561-4a7a-9f9e-d9a6c63564d0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 21:50:10.122037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:10.122037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:10.122037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:10.122037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:10.122037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:10.122037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:10.122037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:10.122037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:10.122037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:10.122037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:10.122037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:10.122037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:50:10.122037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:50:10.122037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:50:10.122037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:50:10.123146 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:50:10.123146 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:50:10.123146 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:50:10.123146 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:50:10.123146 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:10.123146 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:10.123146 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:10.123146 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:10.123146 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:10.123146 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:10.123146 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:10.123146 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:10.123146 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:10.123146 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:10.123146 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:10.123146 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:10.124034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:10.124034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:10.124034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:10.124034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:10.124034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:50:10.124034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:50:10.124034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:50:10.124034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:50:10.124034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:50:10.124034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:50:10.124034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 21:50:10.124034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:10.124681 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 296/582] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jun 7 21:50:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:10.126656 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33883d9-8561-4a7a-9f9e-d9a6c63564d0 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:10.175966 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:10.193131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef94d36c-263c-4a08-9497-3b1b9c2187a4 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:10.243387 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c650969-f60b-4f66-ad34-93bfdf73867f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:10.244622 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c650969-f60b-4f66-ad34-93bfdf73867f None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:10.245104 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c650969-f60b-4f66-ad34-93bfdf73867f None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:10.245572 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c650969-f60b-4f66-ad34-93bfdf73867f None admin] PATH_INFO: `/v3/users/glance` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:10.255038 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c650969-f60b-4f66-ad34-93bfdf73867f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:10.255457 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33883d9-8561-4a7a-9f9e-d9a6c63564d0 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:10.257175 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c650969-f60b-4f66-ad34-93bfdf73867f None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:10.274131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-6c650969-f60b-4f66-ad34-93bfdf73867f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 07 21:50:10.274131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:10.274131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:10.274131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:10.274131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:10.274131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:10.274131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:10.274131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:10.274131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:10.274131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:10.274131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:10.274131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:10.274131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 21:50:10.274131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 21:50:10.274131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 21:50:10.274131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 21:50:10.275137 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:10.275137 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:10.275137 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 21:50:10.275137 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:10.275137 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 21:50:10.275137 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:10.275137 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:10.275137 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:10.275137 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:10.275137 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:10.275137 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:10.275137 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:10.275137 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:10.275137 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:10.275137 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:10.275137 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:10.275137 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:10.275137 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:10.276131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:10.276131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:10.276131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 21:50:10.276131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 21:50:10.276131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 21:50:10.276131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 21:50:10.276131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 21:50:10.276131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 21:50:10.276131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 07 21:50:10.276131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:10.278138 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 287/583] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:50:10 2022] GET /identity/v3/users/glance => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:10.279175 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c650969-f60b-4f66-ad34-93bfdf73867f None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:10.298429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc8845-99da-47af-a8e9-b333c5f49b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:10.299111 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc8845-99da-47af-a8e9-b333c5f49b44 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:10.299791 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc8845-99da-47af-a8e9-b333c5f49b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:10.300083 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc8845-99da-47af-a8e9-b333c5f49b44 None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:10.301145 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cc8845-99da-47af-a8e9-b333c5f49b44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:10.301461 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cc8845-99da-47af-a8e9-b333c5f49b44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:10.302858 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cc8845-99da-47af-a8e9-b333c5f49b44 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:10.320268 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 297/584] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:10.321144 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc8845-99da-47af-a8e9-b333c5f49b44 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:10.335733 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:10.397190 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c650969-f60b-4f66-ad34-93bfdf73867f None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:10.444704 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4991be10-489d-4f2d-a39f-1ecf106166fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:10.447011 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:10.447215 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991be10-489d-4f2d-a39f-1ecf106166fe None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:10.448585 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991be10-489d-4f2d-a39f-1ecf106166fe None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:10.449070 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991be10-489d-4f2d-a39f-1ecf106166fe None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:10.450060 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4991be10-489d-4f2d-a39f-1ecf106166fe None admin] RBAC: Adding query filter params (name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:10.452001 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc8845-99da-47af-a8e9-b333c5f49b44 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:10.452138 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4991be10-489d-4f2d-a39f-1ecf106166fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:10.452341 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4991be10-489d-4f2d-a39f-1ecf106166fe None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:10.464985 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 288/585] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Jun 7 21:50:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:10.466352 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4991be10-489d-4f2d-a39f-1ecf106166fe None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:10.499832 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a38771-4f33-4106-9852-5bad953d0bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:10.500504 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a38771-4f33-4106-9852-5bad953d0bd2 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:10.500783 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a38771-4f33-4106-9852-5bad953d0bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:10.501051 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a38771-4f33-4106-9852-5bad953d0bd2 None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:10.501879 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a38771-4f33-4106-9852-5bad953d0bd2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:10.502187 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a38771-4f33-4106-9852-5bad953d0bd2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:10.503773 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a38771-4f33-4106-9852-5bad953d0bd2 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:10.522107 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 298/586] 10.222.0.13 () {56 vars in 1137 bytes} [Tue Jun 7 21:50:10 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:10.522904 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a38771-4f33-4106-9852-5bad953d0bd2 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:10.594364 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4991be10-489d-4f2d-a39f-1ecf106166fe None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:10.628619 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:10.630697 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a38771-4f33-4106-9852-5bad953d0bd2 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:10.632845 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ec2d64-68d2-4e3b-a1aa-a8cb6bb55a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:10.633951 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec2d64-68d2-4e3b-a1aa-a8cb6bb55a68 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:10.634514 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec2d64-68d2-4e3b-a1aa-a8cb6bb55a68 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:10.635012 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec2d64-68d2-4e3b-a1aa-a8cb6bb55a68 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:10.644369 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ec2d64-68d2-4e3b-a1aa-a8cb6bb55a68 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:10.646162 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ec2d64-68d2-4e3b-a1aa-a8cb6bb55a68 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:10.661652 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-17ec2d64-68d2-4e3b-a1aa-a8cb6bb55a68 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 21:50:10.661652 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:10.661652 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:10.661652 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:10.661652 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:10.661652 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:10.661652 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:10.661652 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:10.661652 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:10.661652 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:10.661652 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:10.661652 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:10.661652 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 21:50:10.661652 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 21:50:10.661652 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 21:50:10.661652 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 21:50:10.662623 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:10.662623 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:10.662623 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 21:50:10.662623 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:10.662623 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 21:50:10.662623 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:10.662623 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:10.662623 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:10.662623 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:10.662623 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:10.662623 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:10.662623 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:10.662623 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:10.662623 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:10.662623 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:10.662623 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:10.662623 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:10.662623 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:10.663986 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:10.663986 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:10.663986 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 21:50:10.663986 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 21:50:10.663986 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 21:50:10.663986 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 21:50:10.663986 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 21:50:10.663986 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 21:50:10.663986 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 21:50:10.663986 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:10.666467 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 289/587] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:50:10 2022] GET /identity/v3/users/cinder => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:10.667630 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ec2d64-68d2-4e3b-a1aa-a8cb6bb55a68 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:10.668940 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b019d58a-299f-4f71-93dd-d3e9d25846bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:10.669743 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019d58a-299f-4f71-93dd-d3e9d25846bd None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:10.670020 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019d58a-299f-4f71-93dd-d3e9d25846bd None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:10.670647 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019d58a-299f-4f71-93dd-d3e9d25846bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:10.682349 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b019d58a-299f-4f71-93dd-d3e9d25846bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:10.683944 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b019d58a-299f-4f71-93dd-d3e9d25846bd None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:10.703310 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-b019d58a-299f-4f71-93dd-d3e9d25846bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:10.703310 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:10.703310 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:10.703310 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:10.703310 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:10.703310 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:10.703310 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:10.703310 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:10.703310 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:10.703310 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:10.703310 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:10.703310 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:10.703310 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:10.703310 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:10.703310 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:10.704287 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:10.704287 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:10.704287 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:10.704287 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:10.704287 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:10.704287 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:10.704287 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:10.704287 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:10.704287 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:10.704287 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:10.704287 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:10.704287 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:10.704287 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:10.704287 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:10.704287 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:10.704287 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:10.704287 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:10.705165 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:10.705165 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:10.705165 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:10.705165 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:10.705165 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 299/588] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:10.705991 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b019d58a-299f-4f71-93dd-d3e9d25846bd None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:10.719208 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:10.784688 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ec2d64-68d2-4e3b-a1aa-a8cb6bb55a68 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:10.822714 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c9ef1c-fe3b-4fd2-b8fc-9238b9fa333c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:10.823950 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c9ef1c-fe3b-4fd2-b8fc-9238b9fa333c None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:10.824461 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c9ef1c-fe3b-4fd2-b8fc-9238b9fa333c None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:10.824695 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b019d58a-299f-4f71-93dd-d3e9d25846bd None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:10.825153 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c9ef1c-fe3b-4fd2-b8fc-9238b9fa333c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:10.837555 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c9ef1c-fe3b-4fd2-b8fc-9238b9fa333c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:10.839468 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c9ef1c-fe3b-4fd2-b8fc-9238b9fa333c None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:10.858540 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-10c9ef1c-fe3b-4fd2-b8fc-9238b9fa333c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:10.858540 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:10.858540 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:10.858540 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:10.858540 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:10.858540 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:10.858540 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:10.858540 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:10.858540 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:10.858540 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:10.858540 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:10.858540 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:10.858540 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:10.858540 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:10.858540 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:10.859499 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:10.859499 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:10.859499 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:10.859499 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:10.859499 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:10.859499 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:10.859499 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:10.859499 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:10.859499 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:10.859499 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:10.859499 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:10.859499 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:10.859499 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:10.859499 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:10.859499 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:10.859499 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:10.859499 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:10.860480 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:10.860480 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:10.860480 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:10.860480 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:10.861164 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 290/589] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:10.862354 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c9ef1c-fe3b-4fd2-b8fc-9238b9fa333c None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:10.864610 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44143eb6-4780-418b-95c6-c943bfe9888f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:10.865593 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44143eb6-4780-418b-95c6-c943bfe9888f None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:10.865885 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44143eb6-4780-418b-95c6-c943bfe9888f None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:10.866158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44143eb6-4780-418b-95c6-c943bfe9888f None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:10.867156 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44143eb6-4780-418b-95c6-c943bfe9888f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:10.867516 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44143eb6-4780-418b-95c6-c943bfe9888f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:10.869031 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44143eb6-4780-418b-95c6-c943bfe9888f None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:10.888542 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 300/590] 10.222.0.13 () {56 vars in 1137 bytes} [Tue Jun 7 21:50:10 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:10.889550 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44143eb6-4780-418b-95c6-c943bfe9888f None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:10.892542 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:10.959833 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:10.973032 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c9ef1c-fe3b-4fd2-b8fc-9238b9fa333c None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:10.994722 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44143eb6-4780-418b-95c6-c943bfe9888f None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:11.008729 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c833f995-5635-44eb-acb3-729eadd9bb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:11.009745 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833f995-5635-44eb-acb3-729eadd9bb48 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:11.010239 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833f995-5635-44eb-acb3-729eadd9bb48 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:11.010722 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833f995-5635-44eb-acb3-729eadd9bb48 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:11.011753 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c833f995-5635-44eb-acb3-729eadd9bb48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:11.012272 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c833f995-5635-44eb-acb3-729eadd9bb48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:11.013842 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c833f995-5635-44eb-acb3-729eadd9bb48 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:11.026779 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 291/591] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:11.028178 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c833f995-5635-44eb-acb3-729eadd9bb48 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:11.029575 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad69f2e-a27c-4d15-bc44-77ed0fa2afac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:11.030232 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad69f2e-a27c-4d15-bc44-77ed0fa2afac None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:11.030724 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad69f2e-a27c-4d15-bc44-77ed0fa2afac None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:11.030910 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad69f2e-a27c-4d15-bc44-77ed0fa2afac None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:11.031629 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad69f2e-a27c-4d15-bc44-77ed0fa2afac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:11.031935 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad69f2e-a27c-4d15-bc44-77ed0fa2afac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:11.033298 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad69f2e-a27c-4d15-bc44-77ed0fa2afac None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:11.046680 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 301/592] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:11.047485 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad69f2e-a27c-4d15-bc44-77ed0fa2afac None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:11.118266 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:11.138026 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c833f995-5635-44eb-acb3-729eadd9bb48 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:11.156400 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad69f2e-a27c-4d15-bc44-77ed0fa2afac None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:11.176340 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9339e43f-a2a2-4764-8cd5-3cee3047f84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:11.177553 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9339e43f-a2a2-4764-8cd5-3cee3047f84a None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:11.178180 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9339e43f-a2a2-4764-8cd5-3cee3047f84a None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:11.178670 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9339e43f-a2a2-4764-8cd5-3cee3047f84a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:11.184846 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:11.192435 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9339e43f-a2a2-4764-8cd5-3cee3047f84a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:11.194053 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3171f4c6-bd34-4576-b4a2-92b401de9b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:11.194562 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9339e43f-a2a2-4764-8cd5-3cee3047f84a None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:11.194850 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3171f4c6-bd34-4576-b4a2-92b401de9b02 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:11.195134 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3171f4c6-bd34-4576-b4a2-92b401de9b02 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:11.195436 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3171f4c6-bd34-4576-b4a2-92b401de9b02 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:11.203085 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3171f4c6-bd34-4576-b4a2-92b401de9b02 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:11.204567 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3171f4c6-bd34-4576-b4a2-92b401de9b02 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:11.216161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-9339e43f-a2a2-4764-8cd5-3cee3047f84a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:11.216161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:11.216161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:11.216161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:11.216161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:11.216161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:11.216161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:11.216161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:11.216161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:11.216161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:11.216161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:11.216161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:11.216161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:11.216161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:11.216161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:11.217090 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:11.217090 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:11.217090 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:11.217090 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:11.217090 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:11.217090 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:11.217090 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:11.217090 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:11.217090 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:11.217090 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:11.217090 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:11.217090 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:11.217090 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:11.217090 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:11.217090 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:11.217090 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:11.217090 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:11.217983 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:11.217983 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:11.217983 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:11.217983 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:11.219968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-3171f4c6-bd34-4576-b4a2-92b401de9b02 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 21:50:11.219968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:11.219968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:11.219968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:11.219968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:11.219968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:11.219968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:11.219968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:11.219968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:11.219968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:11.219968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:11.219968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:11.219968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 21:50:11.219968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 21:50:11.219968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 21:50:11.219968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 21:50:11.220899 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:11.220899 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:11.220899 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 21:50:11.220899 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:11.220899 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 21:50:11.220899 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:11.220899 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:11.220899 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:11.220899 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:11.220899 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:11.220899 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:11.220899 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:11.220899 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:11.220899 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:11.220899 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:11.220899 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:11.220899 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:11.220899 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:11.222455 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:11.222455 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:11.222455 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 21:50:11.222455 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 21:50:11.222455 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 21:50:11.222455 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 21:50:11.222455 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 21:50:11.222455 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 21:50:11.222455 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 21:50:11.222455 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:11.222455 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 302/593] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jun 7 21:50:11 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:11.223072 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3171f4c6-bd34-4576-b4a2-92b401de9b02 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:11.224057 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 292/594] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:11.226293 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9339e43f-a2a2-4764-8cd5-3cee3047f84a None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:11.327117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3171f4c6-bd34-4576-b4a2-92b401de9b02 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:11.338878 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9339e43f-a2a2-4764-8cd5-3cee3047f84a None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:11.363178 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602fd8e1-ee25-4495-ac35-3fa3eb774d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:11.364000 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602fd8e1-ee25-4495-ac35-3fa3eb774d2b None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:11.364279 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602fd8e1-ee25-4495-ac35-3fa3eb774d2b None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:11.364552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602fd8e1-ee25-4495-ac35-3fa3eb774d2b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:11.372760 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602fd8e1-ee25-4495-ac35-3fa3eb774d2b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:11.376182 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602fd8e1-ee25-4495-ac35-3fa3eb774d2b None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:11.381587 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec93b55-7372-4648-9590-77f1f46ab9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:11.383163 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec93b55-7372-4648-9590-77f1f46ab9c2 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:11.384046 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec93b55-7372-4648-9590-77f1f46ab9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:11.384551 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec93b55-7372-4648-9590-77f1f46ab9c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:11.385524 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec93b55-7372-4648-9590-77f1f46ab9c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:11.386054 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec93b55-7372-4648-9590-77f1f46ab9c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:11.388202 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec93b55-7372-4648-9590-77f1f46ab9c2 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:11.391352 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:11.393855 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-602fd8e1-ee25-4495-ac35-3fa3eb774d2b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:11.393855 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:11.393855 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:11.393855 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:11.393855 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:11.393855 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:11.393855 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:11.393855 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:11.393855 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:11.393855 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:11.393855 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:11.393855 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:11.393855 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 21:50:11.393855 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 21:50:11.393855 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 21:50:11.394816 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 21:50:11.394816 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:11.394816 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:11.394816 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:11.394816 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:11.394816 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:11.394816 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:11.394816 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:11.394816 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:11.394816 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:11.394816 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:11.394816 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:11.394816 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:11.394816 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:11.394816 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:11.394816 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:11.394816 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:11.396138 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 21:50:11.396138 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 21:50:11.396138 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 21:50:11.396138 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 21:50:11.396138 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 21:50:11.396138 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 21:50:11.396138 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:11.396138 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:11.396138 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 303/595] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:50:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:11.397314 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602fd8e1-ee25-4495-ac35-3fa3eb774d2b None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:11.408182 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 293/596] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:50:11.409640 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec93b55-7372-4648-9590-77f1f46ab9c2 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:11.451645 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:11.508720 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602fd8e1-ee25-4495-ac35-3fa3eb774d2b None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:11.524682 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec93b55-7372-4648-9590-77f1f46ab9c2 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:11.545035 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0c0dd6-43e3-42c9-9ae2-1e2f6e305274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:11.545681 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c0dd6-43e3-42c9-9ae2-1e2f6e305274 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:11.545966 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c0dd6-43e3-42c9-9ae2-1e2f6e305274 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:11.546240 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c0dd6-43e3-42c9-9ae2-1e2f6e305274 None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:11.547069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0c0dd6-43e3-42c9-9ae2-1e2f6e305274 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:11.547401 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0c0dd6-43e3-42c9-9ae2-1e2f6e305274 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:11.548831 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0c0dd6-43e3-42c9-9ae2-1e2f6e305274 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:11.566244 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ae483-9c7c-4ea4-ab5d-a149a3e90973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:11.567247 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ae483-9c7c-4ea4-ab5d-a149a3e90973 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:11.567772 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ae483-9c7c-4ea4-ab5d-a149a3e90973 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:11.568273 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ae483-9c7c-4ea4-ab5d-a149a3e90973 None admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:11.569081 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 304/597] 10.222.0.13 () {56 vars in 1139 bytes} [Tue Jun 7 21:50:11 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:11.569561 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90ae483-9c7c-4ea4-ab5d-a149a3e90973 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:11.569983 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0c0dd6-43e3-42c9-9ae2-1e2f6e305274 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:11.570328 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90ae483-9c7c-4ea4-ab5d-a149a3e90973 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:11.572173 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90ae483-9c7c-4ea4-ab5d-a149a3e90973 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:11.586889 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 294/598] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jun 7 21:50:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:11.588171 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ae483-9c7c-4ea4-ab5d-a149a3e90973 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:11.650825 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:11.684613 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0c0dd6-43e3-42c9-9ae2-1e2f6e305274 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:11.693100 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:11.701937 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ae483-9c7c-4ea4-ab5d-a149a3e90973 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:11.723861 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05f5490-45d6-4228-8da3-1acc52367692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:11.724835 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f5490-45d6-4228-8da3-1acc52367692 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:11.725119 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f5490-45d6-4228-8da3-1acc52367692 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:11.725390 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f5490-45d6-4228-8da3-1acc52367692 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:11.735798 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05f5490-45d6-4228-8da3-1acc52367692 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:11.738580 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05f5490-45d6-4228-8da3-1acc52367692 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:11.749625 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a949784b-1bb5-4de9-8f7a-8cfc9d9981d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:11.751503 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a949784b-1bb5-4de9-8f7a-8cfc9d9981d5 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:11.752880 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a949784b-1bb5-4de9-8f7a-8cfc9d9981d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:11.753619 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a949784b-1bb5-4de9-8f7a-8cfc9d9981d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:11.759875 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-d05f5490-45d6-4228-8da3-1acc52367692 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:11.759875 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:11.759875 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:11.759875 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:11.759875 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:11.759875 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:11.759875 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:11.759875 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:11.759875 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:11.759875 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:11.759875 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:11.759875 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:11.759875 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 21:50:11.759875 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 21:50:11.759875 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 21:50:11.760908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 21:50:11.760908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:11.760908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:11.760908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:11.760908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:11.760908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:11.760908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:11.760908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:11.760908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:11.760908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:11.760908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:11.760908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:11.760908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:11.760908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:11.760908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:11.760908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:11.760908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:11.762041 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 21:50:11.762041 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 21:50:11.762041 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 21:50:11.762041 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 21:50:11.762041 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 21:50:11.762041 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 21:50:11.762041 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:11.762041 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:11.766822 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 305/599] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:50:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:11.768612 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05f5490-45d6-4228-8da3-1acc52367692 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:11.776960 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a949784b-1bb5-4de9-8f7a-8cfc9d9981d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:11.778792 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a949784b-1bb5-4de9-8f7a-8cfc9d9981d5 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:11.800407 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-a949784b-1bb5-4de9-8f7a-8cfc9d9981d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:11.800407 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:11.800407 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:11.800407 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:11.800407 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:11.800407 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:11.800407 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:11.800407 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:11.800407 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:11.800407 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:11.800407 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:11.800407 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:11.800407 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:11.800407 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:11.800407 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:11.801664 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:11.801664 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:11.801664 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:11.801664 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:11.801664 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:11.801664 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:11.801664 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:11.801664 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:11.801664 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:11.801664 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:11.801664 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:11.801664 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:11.801664 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:11.801664 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:11.801664 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:11.801664 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:11.801664 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:11.802748 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:11.802748 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:11.802748 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:11.802748 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:11.804846 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 295/600] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:11.805784 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a949784b-1bb5-4de9-8f7a-8cfc9d9981d5 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:11.879684 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05f5490-45d6-4228-8da3-1acc52367692 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:11.916892 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ebe942-ac10-4f70-b82c-f88ff89d5bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:11.917856 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a949784b-1bb5-4de9-8f7a-8cfc9d9981d5 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:11.917997 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ebe942-ac10-4f70-b82c-f88ff89d5bbd None admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:11.918381 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ebe942-ac10-4f70-b82c-f88ff89d5bbd None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:11.918753 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ebe942-ac10-4f70-b82c-f88ff89d5bbd None admin] PATH_INFO: `/v3/projects/0219dbda54274fdc9bdccbd960c30c6b/users/96987b04471944bc86ab44ff9936c732/roles/2c7f73c184d0406bbf2899465953342b` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:11.927273 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:11.938037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ebe942-ac10-4f70-b82c-f88ff89d5bbd None admin] RBAC: Authorizing `identity:create_grant(project_id=0219dbda54274fdc9bdccbd960c30c6b, user_id=96987b04471944bc86ab44ff9936c732, role_id=2c7f73c184d0406bbf2899465953342b)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:11.940754 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ebe942-ac10-4f70-b82c-f88ff89d5bbd None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:11.961706 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 306/601] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jun 7 21:50:11 2022] PUT /identity/v3/projects/0219dbda54274fdc9bdccbd960c30c6b/users/96987b04471944bc86ab44ff9936c732/roles/2c7f73c184d0406bbf2899465953342b => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 21:50:11.963080 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ebe942-ac10-4f70-b82c-f88ff89d5bbd None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:11.967397 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ed5797-546d-47e8-b1a1-99fb5dde451e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:11.969155 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed5797-546d-47e8-b1a1-99fb5dde451e None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:11.969446 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed5797-546d-47e8-b1a1-99fb5dde451e None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:11.969728 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed5797-546d-47e8-b1a1-99fb5dde451e None admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:11.970573 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ed5797-546d-47e8-b1a1-99fb5dde451e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:11.970880 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ed5797-546d-47e8-b1a1-99fb5dde451e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:11.973552 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ed5797-546d-47e8-b1a1-99fb5dde451e None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:11.999549 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:12.008278 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 296/602] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jun 7 21:50:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:12.014176 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ed5797-546d-47e8-b1a1-99fb5dde451e None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:12.199135 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ebe942-ac10-4f70-b82c-f88ff89d5bbd None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:12.219818 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ed5797-546d-47e8-b1a1-99fb5dde451e None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:12.249142 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dac0e2-0558-4b78-a68c-785fe1b86852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:12.249999 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dac0e2-0558-4b78-a68c-785fe1b86852 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:12.250284 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dac0e2-0558-4b78-a68c-785fe1b86852 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:12.250822 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dac0e2-0558-4b78-a68c-785fe1b86852 None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:12.251996 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dac0e2-0558-4b78-a68c-785fe1b86852 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:12.252373 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dac0e2-0558-4b78-a68c-785fe1b86852 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:12.255040 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dac0e2-0558-4b78-a68c-785fe1b86852 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:12.263012 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da87d128-82d0-41c3-bb3c-2e1766e538a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:12.264010 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87d128-82d0-41c3-bb3c-2e1766e538a0 None admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:12.264409 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87d128-82d0-41c3-bb3c-2e1766e538a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:12.264694 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87d128-82d0-41c3-bb3c-2e1766e538a0 None admin] PATH_INFO: `/v3/projects/0219dbda54274fdc9bdccbd960c30c6b/users/ea8a1e2cf1da4f87b8d4e16f525fc759/roles/2c7f73c184d0406bbf2899465953342b` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:12.270227 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 307/603] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:12.276356 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dac0e2-0558-4b78-a68c-785fe1b86852 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:12.280961 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da87d128-82d0-41c3-bb3c-2e1766e538a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=0219dbda54274fdc9bdccbd960c30c6b, user_id=ea8a1e2cf1da4f87b8d4e16f525fc759, role_id=2c7f73c184d0406bbf2899465953342b)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:12.282477 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da87d128-82d0-41c3-bb3c-2e1766e538a0 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:12.299862 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 297/604] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jun 7 21:50:12 2022] PUT /identity/v3/projects/0219dbda54274fdc9bdccbd960c30c6b/users/ea8a1e2cf1da4f87b8d4e16f525fc759/roles/2c7f73c184d0406bbf2899465953342b => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 21:50:12.356746 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf234d-6cbf-4064-abe8-939aa59bcddf None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:12.357275 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf234d-6cbf-4064-abe8-939aa59bcddf None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:12.357669 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf234d-6cbf-4064-abe8-939aa59bcddf None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:12.359574 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 298/605] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:12.366380 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca24c7-c92a-4d8a-9818-7f7eb8146e76 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:12.366796 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca24c7-c92a-4d8a-9818-7f7eb8146e76 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:12.367175 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca24c7-c92a-4d8a-9818-7f7eb8146e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:12.384615 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:12.400594 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-edca24c7-c92a-4d8a-9818-7f7eb8146e76 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:12.401977 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-edca24c7-c92a-4d8a-9818-7f7eb8146e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:12.415317 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:12.445159 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dac0e2-0558-4b78-a68c-785fe1b86852 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:12.510776 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 299/606] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 07 21:50:12.515127 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b24003-d3a6-41e0-9a45-d3d940a125f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:12.516014 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b24003-d3a6-41e0-9a45-d3d940a125f7 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:12.516393 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b24003-d3a6-41e0-9a45-d3d940a125f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:12.516767 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b24003-d3a6-41e0-9a45-d3d940a125f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:12.525175 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b24003-d3a6-41e0-9a45-d3d940a125f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:12.526871 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b24003-d3a6-41e0-9a45-d3d940a125f7 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:12.543139 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-e8b24003-d3a6-41e0-9a45-d3d940a125f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:12.543139 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:12.543139 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:12.543139 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:12.543139 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:12.543139 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:12.543139 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:12.543139 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:12.543139 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:12.543139 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:12.543139 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:12.543139 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:12.543139 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 21:50:12.543139 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 21:50:12.543139 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 21:50:12.544300 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 21:50:12.544300 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:12.544300 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:12.544300 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:12.544300 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:12.544300 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:12.544300 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:12.544300 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:12.544300 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:12.544300 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:12.544300 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:12.544300 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:12.544300 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:12.544300 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:12.544300 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:12.544300 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:12.544300 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:12.545192 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 21:50:12.545192 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 21:50:12.545192 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 21:50:12.545192 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 21:50:12.545192 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 21:50:12.545192 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 21:50:12.545192 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:12.545192 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:12.545623 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 308/607] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:50:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:50:12.551677 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edca24c7-c92a-4d8a-9818-7f7eb8146e76 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:12.561513 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b24003-d3a6-41e0-9a45-d3d940a125f7 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:12.681168 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edca24c7-c92a-4d8a-9818-7f7eb8146e76 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:12.695879 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b24003-d3a6-41e0-9a45-d3d940a125f7 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:12.717031 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750750b-9aef-4fba-937e-ddb268cc8764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:12.717799 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750750b-9aef-4fba-937e-ddb268cc8764 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:12.718081 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750750b-9aef-4fba-937e-ddb268cc8764 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:12.718464 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750750b-9aef-4fba-937e-ddb268cc8764 None admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:12.719259 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e750750b-9aef-4fba-937e-ddb268cc8764 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:12.719596 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e750750b-9aef-4fba-937e-ddb268cc8764 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:12.721051 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e750750b-9aef-4fba-937e-ddb268cc8764 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:12.735382 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 300/608] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jun 7 21:50:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:12.737015 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea5209-81c8-4b05-939a-20776166fd0c None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:12.737233 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea5209-81c8-4b05-939a-20776166fd0c None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:12.737359 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea5209-81c8-4b05-939a-20776166fd0c None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:12.740250 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a14f2b-a4c4-4bf4-8c92-d0c8f98f0c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:12.741214 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a14f2b-a4c4-4bf4-8c92-d0c8f98f0c6a None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:12.741654 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a14f2b-a4c4-4bf4-8c92-d0c8f98f0c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:12.742066 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a14f2b-a4c4-4bf4-8c92-d0c8f98f0c6a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:12.742768 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 301/609] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:12.743567 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ea5209-81c8-4b05-939a-20776166fd0c None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:12.745406 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:12.756605 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a14f2b-a4c4-4bf4-8c92-d0c8f98f0c6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:50:12.761113 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:12.770352 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-94a14f2b-a4c4-4bf4-8c92-d0c8f98f0c6a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 21:50:12.770352 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:12.770352 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:12.770352 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:12.770352 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:12.770352 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:12.770352 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:12.770352 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:12.770352 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:12.770352 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:12.770352 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:12.770352 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:12.770352 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:50:12.770352 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:50:12.770352 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:50:12.770352 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:50:12.771388 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:50:12.771388 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:50:12.771388 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:50:12.771388 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:50:12.771388 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:12.771388 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:12.771388 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:12.771388 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:12.771388 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:12.771388 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:12.771388 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:12.771388 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:12.771388 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:12.771388 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:12.771388 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:12.771388 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:12.772240 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:12.772240 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:12.772240 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:12.772240 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:12.772240 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:50:12.772240 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:50:12.772240 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:50:12.772240 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:50:12.772240 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:50:12.772240 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:50:12.772240 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 21:50:12.772240 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:12.775365 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 309/610] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jun 7 21:50:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:12.777097 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e07b13-f3dd-4c3c-be7f-edf34d25f523 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:12.777766 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e07b13-f3dd-4c3c-be7f-edf34d25f523 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:12.778205 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e07b13-f3dd-4c3c-be7f-edf34d25f523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:12.804876 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-28e07b13-f3dd-4c3c-be7f-edf34d25f523 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:12.806181 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-28e07b13-f3dd-4c3c-be7f-edf34d25f523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:12.873917 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ea5209-81c8-4b05-939a-20776166fd0c None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:12.879407 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 310/611] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:50:12.880268 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e07b13-f3dd-4c3c-be7f-edf34d25f523 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:12.934480 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fb45ee-d423-45f2-8c01-4265075a71ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:12.935510 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb45ee-d423-45f2-8c01-4265075a71ae None admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:12.935795 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb45ee-d423-45f2-8c01-4265075a71ae None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:12.936071 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb45ee-d423-45f2-8c01-4265075a71ae None admin] PATH_INFO: `/v3/projects/0219dbda54274fdc9bdccbd960c30c6b/users/10242251e6284217bfb79c67b1567f75/roles/2c7f73c184d0406bbf2899465953342b` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:12.952432 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fb45ee-d423-45f2-8c01-4265075a71ae None admin] RBAC: Authorizing `identity:create_grant(project_id=0219dbda54274fdc9bdccbd960c30c6b, user_id=10242251e6284217bfb79c67b1567f75, role_id=2c7f73c184d0406bbf2899465953342b)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:12.954183 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fb45ee-d423-45f2-8c01-4265075a71ae None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:12.970719 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 302/612] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jun 7 21:50:12 2022] PUT /identity/v3/projects/0219dbda54274fdc9bdccbd960c30c6b/users/10242251e6284217bfb79c67b1567f75/roles/2c7f73c184d0406bbf2899465953342b => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 21:50:12.976288 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fb45ee-d423-45f2-8c01-4265075a71ae None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:12.992340 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e07b13-f3dd-4c3c-be7f-edf34d25f523 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:13.022230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:13.032348 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a2b50a-dc2d-41a2-ae21-ac3af8209dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:13.033304 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a2b50a-dc2d-41a2-ae21-ac3af8209dd5 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:13.034698 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a2b50a-dc2d-41a2-ae21-ac3af8209dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:13.035281 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a2b50a-dc2d-41a2-ae21-ac3af8209dd5 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:13.035842 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a2b50a-dc2d-41a2-ae21-ac3af8209dd5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:13.036546 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a2b50a-dc2d-41a2-ae21-ac3af8209dd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:13.038182 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a2b50a-dc2d-41a2-ae21-ac3af8209dd5 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:13.047902 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 311/613] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Jun 7 21:50:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:13.053621 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a2b50a-dc2d-41a2-ae21-ac3af8209dd5 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:13.103858 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:13.195597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fb45ee-d423-45f2-8c01-4265075a71ae None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:13.247479 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a2b50a-dc2d-41a2-ae21-ac3af8209dd5 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:13.266428 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d828a88c-cc36-4e51-ac3c-2c77364111b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:13.267236 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828a88c-cc36-4e51-ac3c-2c77364111b9 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:13.267545 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828a88c-cc36-4e51-ac3c-2c77364111b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:13.267819 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828a88c-cc36-4e51-ac3c-2c77364111b9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:13.280411 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d828a88c-cc36-4e51-ac3c-2c77364111b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:50:13.285979 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19001886-3080-477d-b9f9-c899211ab92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:13.286783 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19001886-3080-477d-b9f9-c899211ab92d None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:13.287069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19001886-3080-477d-b9f9-c899211ab92d None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:13.287379 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19001886-3080-477d-b9f9-c899211ab92d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:13.288891 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-d828a88c-cc36-4e51-ac3c-2c77364111b9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 21:50:13.288891 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:13.288891 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:13.288891 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:13.288891 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:13.288891 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:13.288891 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:13.288891 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:13.288891 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:13.288891 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:13.288891 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:13.288891 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:13.288891 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:50:13.288891 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:50:13.288891 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:50:13.288891 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:50:13.289790 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:50:13.289790 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:50:13.289790 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:50:13.289790 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:50:13.289790 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:13.289790 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:13.289790 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:13.289790 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:13.289790 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:13.289790 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:13.289790 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:13.289790 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:13.289790 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:13.289790 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:13.289790 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:13.289790 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:13.290616 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:13.290616 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:13.290616 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:13.290616 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:13.290616 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:50:13.290616 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:50:13.290616 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:50:13.290616 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:50:13.290616 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:50:13.290616 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:50:13.290616 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 21:50:13.290616 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:13.290616 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 303/614] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jun 7 21:50:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:13.301926 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d828a88c-cc36-4e51-ac3c-2c77364111b9 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:13.304647 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19001886-3080-477d-b9f9-c899211ab92d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:13.306374 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19001886-3080-477d-b9f9-c899211ab92d None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:13.329479 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-19001886-3080-477d-b9f9-c899211ab92d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:13.329479 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:13.329479 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:13.329479 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:13.329479 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:13.329479 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:13.329479 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:13.329479 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:13.329479 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:13.329479 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:13.329479 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:13.329479 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:13.329479 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:13.329479 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:13.329479 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:13.331049 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:13.331049 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:13.331049 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:13.331049 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:13.331049 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:13.331049 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:13.331049 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:13.331049 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:13.331049 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:13.331049 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:13.331049 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:13.331049 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:13.331049 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:13.331049 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:13.331049 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:13.331049 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:13.331049 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:13.332113 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:13.332113 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:13.332113 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:13.332113 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:13.334330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 312/615] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:13.339910 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19001886-3080-477d-b9f9-c899211ab92d None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:13.389000 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:13.420856 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d828a88c-cc36-4e51-ac3c-2c77364111b9 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:13.456670 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:13.464503 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2cd8b6-037f-4483-9442-0a3e64b0875c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:13.465358 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2cd8b6-037f-4483-9442-0a3e64b0875c None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:13.465640 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2cd8b6-037f-4483-9442-0a3e64b0875c None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:13.465925 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2cd8b6-037f-4483-9442-0a3e64b0875c None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:13.467944 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19001886-3080-477d-b9f9-c899211ab92d None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:13.470034 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2cd8b6-037f-4483-9442-0a3e64b0875c None admin] RBAC: Adding query filter params (name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:13.470357 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2cd8b6-037f-4483-9442-0a3e64b0875c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:13.472420 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2cd8b6-037f-4483-9442-0a3e64b0875c None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:13.489696 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 304/616] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Jun 7 21:50:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:13.496714 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2cd8b6-037f-4483-9442-0a3e64b0875c None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:13.539615 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7599f46-bb7f-4248-8170-ec9466cfff8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:13.540403 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7599f46-bb7f-4248-8170-ec9466cfff8e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:13.540687 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7599f46-bb7f-4248-8170-ec9466cfff8e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:13.540960 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7599f46-bb7f-4248-8170-ec9466cfff8e None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:13.541688 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7599f46-bb7f-4248-8170-ec9466cfff8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:13.541994 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7599f46-bb7f-4248-8170-ec9466cfff8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:13.543380 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7599f46-bb7f-4248-8170-ec9466cfff8e None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:13.570562 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 313/617] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:13.575261 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7599f46-bb7f-4248-8170-ec9466cfff8e None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:13.681109 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:13.684657 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7599f46-bb7f-4248-8170-ec9466cfff8e None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:13.685487 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2cd8b6-037f-4483-9442-0a3e64b0875c None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:13.725826 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9374e04c-cca6-46b1-af46-b29ff4c21b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:13.726630 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9374e04c-cca6-46b1-af46-b29ff4c21b4c None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:13.730909 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9374e04c-cca6-46b1-af46-b29ff4c21b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:13.731220 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9374e04c-cca6-46b1-af46-b29ff4c21b4c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:13.740904 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9374e04c-cca6-46b1-af46-b29ff4c21b4c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:13.742403 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9374e04c-cca6-46b1-af46-b29ff4c21b4c None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:13.758474 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-9374e04c-cca6-46b1-af46-b29ff4c21b4c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 21:50:13.758474 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:13.758474 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:13.758474 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:13.758474 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:13.758474 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:13.758474 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:13.758474 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:13.758474 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:13.758474 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:13.758474 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:13.758474 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:13.758474 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 21:50:13.758474 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 21:50:13.758474 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 21:50:13.758474 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 21:50:13.759421 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:13.759421 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:13.759421 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 21:50:13.759421 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:13.759421 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 21:50:13.759421 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:13.759421 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:13.759421 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:13.759421 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:13.759421 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:13.759421 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:13.759421 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:13.759421 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:13.759421 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:13.759421 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:13.759421 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:13.759421 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:13.759421 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:13.760354 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:13.760354 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:13.760354 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 21:50:13.760354 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 21:50:13.760354 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 21:50:13.760354 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 21:50:13.760354 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 21:50:13.760354 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 21:50:13.760354 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 21:50:13.760354 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:13.760354 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 314/618] 10.222.0.13 () {56 vars in 1093 bytes} [Tue Jun 7 21:50:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:13.765829 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9374e04c-cca6-46b1-af46-b29ff4c21b4c None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:13.786030 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3582309-8a74-41b7-9cde-8f7626e2f2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:13.786829 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3582309-8a74-41b7-9cde-8f7626e2f2af None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:13.787109 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3582309-8a74-41b7-9cde-8f7626e2f2af None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:13.787414 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3582309-8a74-41b7-9cde-8f7626e2f2af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:13.805604 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3582309-8a74-41b7-9cde-8f7626e2f2af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:13.806972 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3582309-8a74-41b7-9cde-8f7626e2f2af None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:13.848495 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-e3582309-8a74-41b7-9cde-8f7626e2f2af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:13.848495 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:13.848495 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:13.848495 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:13.848495 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:13.848495 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:13.848495 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:13.848495 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:13.848495 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:13.848495 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:13.848495 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:13.848495 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:13.848495 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:13.848495 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:13.848495 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:13.850697 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:13.850697 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:13.850697 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:13.850697 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:13.850697 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:13.850697 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:13.850697 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:13.850697 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:13.850697 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:13.850697 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:13.850697 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:13.850697 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:13.850697 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:13.850697 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:13.850697 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:13.850697 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:13.850697 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:13.852819 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:13.852819 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:13.852819 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:13.852819 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:13.852819 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 305/619] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:13.856798 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3582309-8a74-41b7-9cde-8f7626e2f2af None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:13.880302 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9374e04c-cca6-46b1-af46-b29ff4c21b4c None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:13.913349 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:13.917292 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1880b3d-07b7-42ef-b8db-44afb88bd371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:13.918157 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1880b3d-07b7-42ef-b8db-44afb88bd371 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:13.918594 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1880b3d-07b7-42ef-b8db-44afb88bd371 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:13.919018 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1880b3d-07b7-42ef-b8db-44afb88bd371 None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:13.920152 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1880b3d-07b7-42ef-b8db-44afb88bd371 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:13.920632 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1880b3d-07b7-42ef-b8db-44afb88bd371 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:13.922233 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1880b3d-07b7-42ef-b8db-44afb88bd371 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:13.941417 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 315/620] 10.222.0.13 () {56 vars in 1133 bytes} [Tue Jun 7 21:50:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:13.946948 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1880b3d-07b7-42ef-b8db-44afb88bd371 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:13.966395 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:14.051048 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3582309-8a74-41b7-9cde-8f7626e2f2af None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:14.055175 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1880b3d-07b7-42ef-b8db-44afb88bd371 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:14.093479 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6fb37c-2ab6-4885-9368-478e3fb6a273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:14.094384 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6fb37c-2ab6-4885-9368-478e3fb6a273 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:14.094843 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6fb37c-2ab6-4885-9368-478e3fb6a273 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:14.095290 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6fb37c-2ab6-4885-9368-478e3fb6a273 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:14.120970 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e439f633-3548-4127-8787-a57f392af7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:14.121660 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439f633-3548-4127-8787-a57f392af7cf None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:14.121945 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439f633-3548-4127-8787-a57f392af7cf None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:14.122224 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439f633-3548-4127-8787-a57f392af7cf None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:14.122976 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e439f633-3548-4127-8787-a57f392af7cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:14.123261 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e439f633-3548-4127-8787-a57f392af7cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:14.124664 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e439f633-3548-4127-8787-a57f392af7cf None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:14.149128 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 306/621] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:14.154765 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e439f633-3548-4127-8787-a57f392af7cf None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:14.177107 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6fb37c-2ab6-4885-9368-478e3fb6a273 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:14.178803 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6fb37c-2ab6-4885-9368-478e3fb6a273 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:14.197714 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:14.286713 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 316/622] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:50:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 21:50:14.292713 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6fb37c-2ab6-4885-9368-478e3fb6a273 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:14.354262 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:14.376774 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e439f633-3548-4127-8787-a57f392af7cf None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:14.400314 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6fb37c-2ab6-4885-9368-478e3fb6a273 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:14.419988 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:14.438696 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afd6a86-0547-4e07-99dd-6356b8b53b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:14.440416 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afd6a86-0547-4e07-99dd-6356b8b53b5a None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:14.440953 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afd6a86-0547-4e07-99dd-6356b8b53b5a None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:14.441378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afd6a86-0547-4e07-99dd-6356b8b53b5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:14.452108 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afd6a86-0547-4e07-99dd-6356b8b53b5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:14.453758 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afd6a86-0547-4e07-99dd-6356b8b53b5a None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:14.459029 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2503307b-f7a4-48a1-a8a7-adab0cb1bafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:14.459870 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2503307b-f7a4-48a1-a8a7-adab0cb1bafe None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:14.460183 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2503307b-f7a4-48a1-a8a7-adab0cb1bafe None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:14.460425 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2503307b-f7a4-48a1-a8a7-adab0cb1bafe None admin] PATH_INFO: `/v3/users/swift` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:14.469259 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2503307b-f7a4-48a1-a8a7-adab0cb1bafe None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:14.471129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2503307b-f7a4-48a1-a8a7-adab0cb1bafe None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:14.477841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-3afd6a86-0547-4e07-99dd-6356b8b53b5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:14.477841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:14.477841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:14.477841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:14.477841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:14.477841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:14.477841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:14.477841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:14.477841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:14.477841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:14.477841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:14.477841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:14.477841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:14.477841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:14.477841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:14.478813 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:14.478813 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:14.478813 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:14.478813 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:14.478813 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:14.478813 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:14.478813 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:14.478813 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:14.478813 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:14.478813 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:14.478813 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:14.478813 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:14.478813 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:14.478813 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:14.478813 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:14.478813 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:14.478813 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:14.479763 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:14.479763 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:14.479763 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:14.479763 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:14.488569 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 317/623] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:50:14.496584 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-2503307b-f7a4-48a1-a8a7-adab0cb1bafe None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 21:50:14.496584 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:14.496584 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:14.496584 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:14.496584 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:14.496584 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:14.496584 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:14.496584 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:14.496584 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:14.496584 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:14.496584 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:14.496584 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:14.496584 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 21:50:14.496584 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 21:50:14.496584 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 21:50:14.496584 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 21:50:14.497611 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:14.497611 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:14.497611 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 21:50:14.497611 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:14.497611 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 21:50:14.497611 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:14.497611 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:14.497611 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:14.497611 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:14.497611 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:14.497611 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:14.497611 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:14.497611 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:14.497611 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:14.497611 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:14.497611 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:14.497611 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:14.497611 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:14.498593 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:14.498593 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:14.498593 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 21:50:14.498593 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 21:50:14.498593 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 21:50:14.498593 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 21:50:14.498593 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 21:50:14.498593 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 21:50:14.498593 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 21:50:14.498593 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:14.499148 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afd6a86-0547-4e07-99dd-6356b8b53b5a None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:14.502761 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 307/624] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jun 7 21:50:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:50:14.509066 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2503307b-f7a4-48a1-a8a7-adab0cb1bafe None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:14.659921 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afd6a86-0547-4e07-99dd-6356b8b53b5a None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:14.697183 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a4fd20-9fac-400b-bd3c-8605deb680cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:14.698228 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a4fd20-9fac-400b-bd3c-8605deb680cd None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:14.698747 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a4fd20-9fac-400b-bd3c-8605deb680cd None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:14.699239 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a4fd20-9fac-400b-bd3c-8605deb680cd None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:14.700228 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a4fd20-9fac-400b-bd3c-8605deb680cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:14.700783 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a4fd20-9fac-400b-bd3c-8605deb680cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:14.702377 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a4fd20-9fac-400b-bd3c-8605deb680cd None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:14.709722 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2503307b-f7a4-48a1-a8a7-adab0cb1bafe None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:14.716587 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 318/625] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:14.722829 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a4fd20-9fac-400b-bd3c-8605deb680cd None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:14.755426 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:14.785760 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71167bb5-184a-4a09-93c2-228d2f01c3b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:14.786826 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71167bb5-184a-4a09-93c2-228d2f01c3b5 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:14.787339 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71167bb5-184a-4a09-93c2-228d2f01c3b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:14.787874 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71167bb5-184a-4a09-93c2-228d2f01c3b5 None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:14.788949 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71167bb5-184a-4a09-93c2-228d2f01c3b5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:14.789470 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71167bb5-184a-4a09-93c2-228d2f01c3b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:14.791135 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71167bb5-184a-4a09-93c2-228d2f01c3b5 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:14.830419 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 308/626] 10.222.0.13 () {56 vars in 1135 bytes} [Tue Jun 7 21:50:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:14.836048 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71167bb5-184a-4a09-93c2-228d2f01c3b5 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:14.852738 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:14.909828 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a4fd20-9fac-400b-bd3c-8605deb680cd None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:14.947184 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5179575f-6231-4656-b03f-5258b2459a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:14.947927 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5179575f-6231-4656-b03f-5258b2459a55 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:14.948211 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5179575f-6231-4656-b03f-5258b2459a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:14.948484 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5179575f-6231-4656-b03f-5258b2459a55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:14.960757 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5179575f-6231-4656-b03f-5258b2459a55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:14.962504 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5179575f-6231-4656-b03f-5258b2459a55 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:14.966594 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71167bb5-184a-4a09-93c2-228d2f01c3b5 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:14.977886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-5179575f-6231-4656-b03f-5258b2459a55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:14.977886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:14.977886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:14.977886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:14.977886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:14.977886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:14.977886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:14.977886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:14.977886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:14.977886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:14.977886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:14.977886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:14.977886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 21:50:14.977886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 21:50:14.977886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 21:50:14.978780 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 21:50:14.978780 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:14.978780 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:14.978780 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:14.978780 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:14.978780 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:14.978780 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:14.978780 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:14.978780 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:14.978780 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:14.978780 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:14.978780 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:14.978780 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:14.978780 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:14.978780 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:14.978780 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:14.978780 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:14.980720 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 21:50:14.980720 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 21:50:14.980720 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 21:50:14.980720 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 21:50:14.980720 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 21:50:14.980720 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 21:50:14.980720 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:14.980720 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:14.980720 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 319/627] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:50:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:50:14.980720 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9b122-1aca-4848-a9d2-85b73cfe3f87 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:14.980720 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9b122-1aca-4848-a9d2-85b73cfe3f87 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:14.981351 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9b122-1aca-4848-a9d2-85b73cfe3f87 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:14.982254 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 320/628] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:14.983311 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3716d6-59de-432f-8ed1-1163546f8dd0 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:14.985057 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3716d6-59de-432f-8ed1-1163546f8dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:14.985057 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3716d6-59de-432f-8ed1-1163546f8dd0 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:14.987556 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 321/629] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:14.987556 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3716d6-59de-432f-8ed1-1163546f8dd0 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:15.010971 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ac9a2-3925-4796-ae59-3a3ce608bc73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:15.011972 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ac9a2-3925-4796-ae59-3a3ce608bc73 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:15.012250 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ac9a2-3925-4796-ae59-3a3ce608bc73 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:15.012520 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ac9a2-3925-4796-ae59-3a3ce608bc73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:15.023646 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2ac9a2-3925-4796-ae59-3a3ce608bc73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:15.025088 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2ac9a2-3925-4796-ae59-3a3ce608bc73 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:15.043912 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-6e2ac9a2-3925-4796-ae59-3a3ce608bc73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:15.043912 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:15.043912 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:15.043912 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:15.043912 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:15.043912 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:15.043912 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:15.043912 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:15.043912 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:15.043912 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:15.043912 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:15.043912 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:15.043912 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:15.043912 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:15.043912 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:15.044830 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:15.044830 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:15.044830 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:15.044830 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:15.044830 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:15.044830 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:15.044830 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:15.044830 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:15.044830 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:15.044830 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:15.044830 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:15.044830 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:15.044830 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:15.044830 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:15.044830 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:15.044830 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:15.044830 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:15.045744 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:15.045744 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:15.045744 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:15.045744 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:15.045744 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 309/630] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:15.046264 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959df1c7-7693-49d9-a64d-85492d629735 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:15.046550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959df1c7-7693-49d9-a64d-85492d629735 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:15.046830 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959df1c7-7693-49d9-a64d-85492d629735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:15.066686 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-959df1c7-7693-49d9-a64d-85492d629735 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:15.067887 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-959df1c7-7693-49d9-a64d-85492d629735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:15.088583 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:15.102670 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3716d6-59de-432f-8ed1-1163546f8dd0 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:15.124658 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:15.137001 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 310/631] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:50:15.138233 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c44be-9c89-49cf-95e2-44e0ede09ad3 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:15.138523 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c44be-9c89-49cf-95e2-44e0ede09ad3 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:15.138797 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c44be-9c89-49cf-95e2-44e0ede09ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:15.145625 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c898200-1e2b-4230-a585-5b5e6cb1d4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:15.147222 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c898200-1e2b-4230-a585-5b5e6cb1d4b6 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:15.147628 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c898200-1e2b-4230-a585-5b5e6cb1d4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:15.148002 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c898200-1e2b-4230-a585-5b5e6cb1d4b6 None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:15.148901 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c898200-1e2b-4230-a585-5b5e6cb1d4b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:15.149306 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c898200-1e2b-4230-a585-5b5e6cb1d4b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:15.150841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c898200-1e2b-4230-a585-5b5e6cb1d4b6 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:15.158473 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-0b3c44be-9c89-49cf-95e2-44e0ede09ad3 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:15.161076 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-0b3c44be-9c89-49cf-95e2-44e0ede09ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:15.166194 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 322/632] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jun 7 21:50:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:15.167085 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c898200-1e2b-4230-a585-5b5e6cb1d4b6 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:15.235041 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 311/633] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:50:15.244284 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c44be-9c89-49cf-95e2-44e0ede09ad3 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:15.275424 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c898200-1e2b-4230-a585-5b5e6cb1d4b6 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:15.310516 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a13dc9-c34d-4d4f-af10-6ad4b4141d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:15.311182 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a13dc9-c34d-4d4f-af10-6ad4b4141d43 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:15.311483 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a13dc9-c34d-4d4f-af10-6ad4b4141d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:15.311760 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a13dc9-c34d-4d4f-af10-6ad4b4141d43 None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:15.312486 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a13dc9-c34d-4d4f-af10-6ad4b4141d43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:15.312795 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a13dc9-c34d-4d4f-af10-6ad4b4141d43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:15.314277 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a13dc9-c34d-4d4f-af10-6ad4b4141d43 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:15.332801 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 323/634] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:15.332801 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a13dc9-c34d-4d4f-af10-6ad4b4141d43 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:15.342763 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:15.359142 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c44be-9c89-49cf-95e2-44e0ede09ad3 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:15.380750 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:15.396393 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2608987-0767-42e9-b9c6-fbb53a45cd87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:15.397193 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2608987-0767-42e9-b9c6-fbb53a45cd87 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:15.397479 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2608987-0767-42e9-b9c6-fbb53a45cd87 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:15.397751 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2608987-0767-42e9-b9c6-fbb53a45cd87 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:15.409233 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2608987-0767-42e9-b9c6-fbb53a45cd87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:50:15.418124 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-b2608987-0767-42e9-b9c6-fbb53a45cd87 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 21:50:15.418124 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:15.418124 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:15.418124 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:15.418124 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:15.418124 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:15.418124 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:15.418124 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:15.418124 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:15.418124 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:15.418124 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:15.418124 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:15.418124 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:50:15.418124 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:50:15.418124 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:50:15.418124 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:50:15.419201 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:50:15.419201 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:50:15.419201 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:50:15.419201 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:50:15.419201 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:15.419201 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:15.419201 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:15.419201 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:15.419201 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:15.419201 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:15.419201 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:15.419201 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:15.419201 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:15.419201 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:15.419201 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:15.419201 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:15.420090 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:15.420090 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:15.420090 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:15.420090 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:15.420090 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:50:15.420090 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:50:15.420090 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:50:15.420090 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:50:15.420090 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:50:15.420090 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:50:15.420090 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 21:50:15.420090 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:15.420090 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 312/635] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jun 7 21:50:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:15.420090 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2608987-0767-42e9-b9c6-fbb53a45cd87 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:15.441231 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a13dc9-c34d-4d4f-af10-6ad4b4141d43 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:15.479050 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7748aa-45fe-4353-8342-a413ff0160e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:15.479753 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7748aa-45fe-4353-8342-a413ff0160e4 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:15.480034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7748aa-45fe-4353-8342-a413ff0160e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:15.480305 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7748aa-45fe-4353-8342-a413ff0160e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:15.481151 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7748aa-45fe-4353-8342-a413ff0160e4 None admin] RBAC: Adding query filter params (role.id=2c7f73c184d0406bbf2899465953342b, scope.project.id=0219dbda54274fdc9bdccbd960c30c6b, user.id=dbe0303ea0764347bfb054870ebba41f) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:15.481457 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7748aa-45fe-4353-8342-a413ff0160e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:15.482939 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7748aa-45fe-4353-8342-a413ff0160e4 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:15.499444 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 324/636] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:50:15 2022] GET /identity/v3/role_assignments?user.id=dbe0303ea0764347bfb054870ebba41f&scope.project.id=0219dbda54274fdc9bdccbd960c30c6b&role.id=2c7f73c184d0406bbf2899465953342b => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:15.499444 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7748aa-45fe-4353-8342-a413ff0160e4 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:15.550222 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2608987-0767-42e9-b9c6-fbb53a45cd87 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:15.593427 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e10c438-adcb-43cf-949b-743b154bef48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:15.594233 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10c438-adcb-43cf-949b-743b154bef48 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:15.594511 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10c438-adcb-43cf-949b-743b154bef48 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:15.594784 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10c438-adcb-43cf-949b-743b154bef48 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:15.600323 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:15.608760 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e10c438-adcb-43cf-949b-743b154bef48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:50:15.617739 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-2e10c438-adcb-43cf-949b-743b154bef48 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 21:50:15.617739 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:15.617739 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:15.617739 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:15.617739 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:15.617739 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:15.617739 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:15.617739 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:15.617739 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:15.617739 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:15.617739 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:15.617739 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:15.617739 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:50:15.617739 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:50:15.617739 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:50:15.617739 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:50:15.618757 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:50:15.618757 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:50:15.618757 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:50:15.618757 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:50:15.618757 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:15.618757 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:15.618757 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:15.618757 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:15.618757 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:15.618757 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:15.618757 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:15.618757 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:15.618757 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:15.618757 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:15.618757 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:15.618757 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:15.619631 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:15.619631 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:15.619631 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:15.619631 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:15.619631 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:50:15.619631 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:50:15.619631 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:50:15.619631 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:50:15.619631 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:50:15.619631 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:50:15.619631 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 21:50:15.619631 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:15.619631 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 313/637] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jun 7 21:50:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:15.620345 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e10c438-adcb-43cf-949b-743b154bef48 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:15.640681 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7748aa-45fe-4353-8342-a413ff0160e4 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:15.671880 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:15.692391 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38575bc-6c2d-43c6-9abf-90f68ed923b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:15.694055 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38575bc-6c2d-43c6-9abf-90f68ed923b6 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:15.694781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38575bc-6c2d-43c6-9abf-90f68ed923b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:15.695216 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38575bc-6c2d-43c6-9abf-90f68ed923b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:15.707028 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38575bc-6c2d-43c6-9abf-90f68ed923b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:15.708930 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38575bc-6c2d-43c6-9abf-90f68ed923b6 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:15.728840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-f38575bc-6c2d-43c6-9abf-90f68ed923b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:15.728840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:15.728840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:15.728840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:15.728840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:15.728840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:15.728840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:15.728840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:15.728840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:15.728840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:15.728840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:15.728840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:15.728840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 21:50:15.728840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 21:50:15.728840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 21:50:15.729856 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 21:50:15.729856 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:15.729856 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:15.729856 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:15.729856 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:15.729856 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:15.729856 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:15.729856 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:15.729856 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:15.729856 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:15.729856 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:15.729856 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:15.729856 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:15.729856 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:15.729856 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:15.729856 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:15.729856 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:15.730740 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 21:50:15.730740 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 21:50:15.730740 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 21:50:15.730740 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 21:50:15.730740 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 21:50:15.730740 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 21:50:15.730740 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:15.730740 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:15.735103 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 325/638] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:50:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:15.736025 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38575bc-6c2d-43c6-9abf-90f68ed923b6 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:15.748274 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e10c438-adcb-43cf-949b-743b154bef48 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:15.798456 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba957ec8-35f6-4082-b6eb-ce9bffa39c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:15.800115 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba957ec8-35f6-4082-b6eb-ce9bffa39c02 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:15.800927 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba957ec8-35f6-4082-b6eb-ce9bffa39c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:15.800927 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba957ec8-35f6-4082-b6eb-ce9bffa39c02 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:15.801834 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba957ec8-35f6-4082-b6eb-ce9bffa39c02 None admin] RBAC: Adding query filter params (name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:15.802144 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba957ec8-35f6-4082-b6eb-ce9bffa39c02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:15.803591 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba957ec8-35f6-4082-b6eb-ce9bffa39c02 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:15.813379 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 314/639] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Jun 7 21:50:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:15.813853 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba957ec8-35f6-4082-b6eb-ce9bffa39c02 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:15.869714 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38575bc-6c2d-43c6-9abf-90f68ed923b6 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:15.906886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349bfc2d-f885-491f-8302-f61101bb8749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:15.907602 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349bfc2d-f885-491f-8302-f61101bb8749 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:15.907889 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349bfc2d-f885-491f-8302-f61101bb8749 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:15.908166 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349bfc2d-f885-491f-8302-f61101bb8749 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:15.908955 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349bfc2d-f885-491f-8302-f61101bb8749 None admin] RBAC: Adding query filter params (name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:15.909295 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349bfc2d-f885-491f-8302-f61101bb8749 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:15.910784 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349bfc2d-f885-491f-8302-f61101bb8749 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:15.912358 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:15.920716 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 326/640] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Jun 7 21:50:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:15.921348 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349bfc2d-f885-491f-8302-f61101bb8749 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:15.924673 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba957ec8-35f6-4082-b6eb-ce9bffa39c02 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:15.926353 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:15.963003 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6617646-ec50-4d92-b0cd-3ea1906d0fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:15.964129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6617646-ec50-4d92-b0cd-3ea1906d0fa2 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:15.964129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6617646-ec50-4d92-b0cd-3ea1906d0fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:15.964410 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6617646-ec50-4d92-b0cd-3ea1906d0fa2 None admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:15.965124 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6617646-ec50-4d92-b0cd-3ea1906d0fa2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:15.965432 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6617646-ec50-4d92-b0cd-3ea1906d0fa2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:15.966853 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6617646-ec50-4d92-b0cd-3ea1906d0fa2 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:15.985768 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 315/641] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jun 7 21:50:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:15.985768 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621c880-cb51-436f-a998-4cbb781bc184 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:15.985768 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621c880-cb51-436f-a998-4cbb781bc184 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:15.985768 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621c880-cb51-436f-a998-4cbb781bc184 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:15.985768 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 316/642] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:15.985768 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b621c880-cb51-436f-a998-4cbb781bc184 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:16.038713 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349bfc2d-f885-491f-8302-f61101bb8749 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:16.076237 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0fa673-2ade-448c-8d4c-5ef3d4721d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:16.077022 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fa673-2ade-448c-8d4c-5ef3d4721d37 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:16.077300 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fa673-2ade-448c-8d4c-5ef3d4721d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:16.077847 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fa673-2ade-448c-8d4c-5ef3d4721d37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:16.089709 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0fa673-2ade-448c-8d4c-5ef3d4721d37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:16.091416 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0fa673-2ade-448c-8d4c-5ef3d4721d37 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:16.097614 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b621c880-cb51-436f-a998-4cbb781bc184 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:16.110517 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-df0fa673-2ade-448c-8d4c-5ef3d4721d37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:16.110517 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:16.110517 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:16.110517 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:16.110517 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:16.110517 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:16.110517 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:16.110517 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:16.110517 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:16.110517 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:16.110517 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:16.110517 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:16.110517 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:16.110517 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:16.110517 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:16.111652 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:16.111652 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:16.111652 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:16.111652 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:16.111652 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:16.111652 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:16.111652 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:16.111652 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:16.111652 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:16.111652 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:16.111652 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:16.111652 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:16.111652 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:16.111652 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:16.111652 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:16.111652 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:16.111652 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:16.113189 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:16.113189 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:16.113189 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:16.113189 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:16.113189 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 327/643] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:16.113189 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0fa673-2ade-448c-8d4c-5ef3d4721d37 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:16.143837 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1f94df-96c5-40ca-a4df-2809584821e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:16.144627 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f94df-96c5-40ca-a4df-2809584821e1 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:16.144904 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f94df-96c5-40ca-a4df-2809584821e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:16.145174 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f94df-96c5-40ca-a4df-2809584821e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:16.159659 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1f94df-96c5-40ca-a4df-2809584821e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:16.161186 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1f94df-96c5-40ca-a4df-2809584821e1 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:16.166794 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:16.180200 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-8d1f94df-96c5-40ca-a4df-2809584821e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:16.180200 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:16.180200 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:16.180200 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:16.180200 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:16.180200 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:16.180200 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:16.180200 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:16.180200 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:16.180200 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:16.180200 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:16.180200 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:16.180200 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:16.180200 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:16.180200 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:16.181131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:16.181131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:16.181131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:16.181131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:16.181131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:16.181131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:16.181131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:16.181131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:16.181131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:16.181131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:16.181131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:16.181131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:16.181131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:16.181131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:16.181131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:16.181131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:16.181131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:16.182217 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:16.182217 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:16.182217 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:16.182217 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:16.182217 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 317/644] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:16.183090 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4744b-d715-485a-974e-ede010fc2e7a None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:16.183090 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4744b-d715-485a-974e-ede010fc2e7a None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:16.183090 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4744b-d715-485a-974e-ede010fc2e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:16.200286 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-20b4744b-d715-485a-974e-ede010fc2e7a None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:16.201474 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-20b4744b-d715-485a-974e-ede010fc2e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:16.212056 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:16.234432 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0fa673-2ade-448c-8d4c-5ef3d4721d37 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:16.271878 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 318/645] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:50:16.272827 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d2304-51c1-4bfb-8652-640d32a9c2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:16.273919 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d2304-51c1-4bfb-8652-640d32a9c2b9 None admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:16.274201 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d2304-51c1-4bfb-8652-640d32a9c2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:16.274473 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d2304-51c1-4bfb-8652-640d32a9c2b9 None admin] PATH_INFO: `/v3/projects/0219dbda54274fdc9bdccbd960c30c6b/users/7a1261563b6d4e50a4ffeea6a32bdb01/roles/2c7f73c184d0406bbf2899465953342b` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:16.284415 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b4744b-d715-485a-974e-ede010fc2e7a None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:16.289862 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70d2304-51c1-4bfb-8652-640d32a9c2b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=0219dbda54274fdc9bdccbd960c30c6b, user_id=7a1261563b6d4e50a4ffeea6a32bdb01, role_id=2c7f73c184d0406bbf2899465953342b)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:16.291347 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70d2304-51c1-4bfb-8652-640d32a9c2b9 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:16.315700 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 328/646] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jun 7 21:50:16 2022] PUT /identity/v3/projects/0219dbda54274fdc9bdccbd960c30c6b/users/7a1261563b6d4e50a4ffeea6a32bdb01/roles/2c7f73c184d0406bbf2899465953342b => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 21:50:16.316904 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d2304-51c1-4bfb-8652-640d32a9c2b9 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:16.430947 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b4744b-d715-485a-974e-ede010fc2e7a None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:16.436708 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:16.442445 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d2304-51c1-4bfb-8652-640d32a9c2b9 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:16.495448 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8773a-7ca3-43e3-a34e-d8b1cc93834b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:16.496514 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8773a-7ca3-43e3-a34e-d8b1cc93834b None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:16.497034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8773a-7ca3-43e3-a34e-d8b1cc93834b None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:16.497514 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8773a-7ca3-43e3-a34e-d8b1cc93834b None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:16.498466 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c8773a-7ca3-43e3-a34e-d8b1cc93834b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:16.498981 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c8773a-7ca3-43e3-a34e-d8b1cc93834b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:16.500642 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c8773a-7ca3-43e3-a34e-d8b1cc93834b None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:16.501717 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1c8f11-e922-4b0f-8144-fb6bc7fadd11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:16.503307 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1c8f11-e922-4b0f-8144-fb6bc7fadd11 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:16.504164 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1c8f11-e922-4b0f-8144-fb6bc7fadd11 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:16.504459 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1c8f11-e922-4b0f-8144-fb6bc7fadd11 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:16.505212 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1c8f11-e922-4b0f-8144-fb6bc7fadd11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:16.505561 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1c8f11-e922-4b0f-8144-fb6bc7fadd11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:16.507619 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1c8f11-e922-4b0f-8144-fb6bc7fadd11 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:16.518035 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 329/647] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:16.518260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8773a-7ca3-43e3-a34e-d8b1cc93834b None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:16.522361 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 319/648] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:16.523232 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1c8f11-e922-4b0f-8144-fb6bc7fadd11 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:16.539503 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:16.644556 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1c8f11-e922-4b0f-8144-fb6bc7fadd11 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:16.668231 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8773a-7ca3-43e3-a34e-d8b1cc93834b None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:16.681319 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17945186-29a0-46e9-893e-ced273670b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:16.682432 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17945186-29a0-46e9-893e-ced273670b12 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:16.682933 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17945186-29a0-46e9-893e-ced273670b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:16.683428 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17945186-29a0-46e9-893e-ced273670b12 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:16.691936 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17945186-29a0-46e9-893e-ced273670b12 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:16.693636 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17945186-29a0-46e9-893e-ced273670b12 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:16.709627 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec112b-b0a8-4b1f-9892-26cec243f76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:16.710015 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-17945186-29a0-46e9-893e-ced273670b12 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 07 21:50:16.710015 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:16.710015 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:16.710015 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:16.710015 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:16.710015 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:16.710015 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:16.710015 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:16.710015 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:16.710015 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:16.710015 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:16.710015 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:16.710015 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 21:50:16.710015 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 21:50:16.710015 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 21:50:16.710015 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 21:50:16.710909 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:16.710909 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:16.710909 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 21:50:16.710909 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:16.710909 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 21:50:16.710909 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:16.710909 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:16.710909 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:16.710909 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:16.710909 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:16.710909 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:16.710909 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:16.710909 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:16.710909 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:16.710909 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:16.710909 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:16.710909 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:16.710909 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:16.711903 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:16.711903 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:16.711903 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 21:50:16.711903 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 21:50:16.711903 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 21:50:16.711903 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 21:50:16.711903 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 21:50:16.711903 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 21:50:16.711903 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 07 21:50:16.711903 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:16.712446 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec112b-b0a8-4b1f-9892-26cec243f76c None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:16.712446 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec112b-b0a8-4b1f-9892-26cec243f76c None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:16.712446 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec112b-b0a8-4b1f-9892-26cec243f76c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:16.714736 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 320/649] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:50:16 2022] GET /identity/v3/users/glance => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:16.715836 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17945186-29a0-46e9-893e-ced273670b12 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:16.723768 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec112b-b0a8-4b1f-9892-26cec243f76c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:50:16.733944 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-94ec112b-b0a8-4b1f-9892-26cec243f76c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 21:50:16.733944 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:16.733944 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:16.733944 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:16.733944 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:16.733944 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:16.733944 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:16.733944 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:16.733944 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:16.733944 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:16.733944 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:16.733944 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:16.733944 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:50:16.733944 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:50:16.733944 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:50:16.733944 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:50:16.734914 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:50:16.734914 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:50:16.734914 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:50:16.734914 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:50:16.734914 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:16.734914 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:16.734914 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:16.734914 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:16.734914 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:16.734914 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:16.734914 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:16.734914 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:16.734914 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:16.734914 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:16.734914 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:16.734914 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:16.735810 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:16.735810 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:16.735810 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:16.735810 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:16.735810 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:50:16.735810 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:50:16.735810 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:50:16.735810 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:50:16.735810 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:50:16.735810 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:50:16.735810 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 21:50:16.735810 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:16.739410 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 330/650] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jun 7 21:50:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:16.740471 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec112b-b0a8-4b1f-9892-26cec243f76c None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:16.765657 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:16.810965 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:16.857238 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17945186-29a0-46e9-893e-ced273670b12 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:16.872760 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec112b-b0a8-4b1f-9892-26cec243f76c None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:16.900606 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37348ae2-1ac2-4684-8104-c1cdadabe839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:16.901331 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37348ae2-1ac2-4684-8104-c1cdadabe839 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:16.901615 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37348ae2-1ac2-4684-8104-c1cdadabe839 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:16.901889 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37348ae2-1ac2-4684-8104-c1cdadabe839 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:16.913531 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37348ae2-1ac2-4684-8104-c1cdadabe839 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:16.915284 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2900d4-d856-4ed8-992d-5ec38de8576e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:16.915748 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37348ae2-1ac2-4684-8104-c1cdadabe839 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:16.916173 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2900d4-d856-4ed8-992d-5ec38de8576e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:16.916543 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2900d4-d856-4ed8-992d-5ec38de8576e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:16.916873 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2900d4-d856-4ed8-992d-5ec38de8576e None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:16.917768 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2900d4-d856-4ed8-992d-5ec38de8576e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:16.918142 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2900d4-d856-4ed8-992d-5ec38de8576e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:16.920306 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2900d4-d856-4ed8-992d-5ec38de8576e None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:16.934222 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-37348ae2-1ac2-4684-8104-c1cdadabe839 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 21:50:16.934222 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:16.934222 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:16.934222 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:16.934222 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:16.934222 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:16.934222 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:16.934222 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:16.934222 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:16.934222 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:16.934222 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:16.934222 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:16.934222 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 21:50:16.934222 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 21:50:16.934222 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 21:50:16.934222 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 21:50:16.935234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:16.935234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:16.935234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 21:50:16.935234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:16.935234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 21:50:16.935234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:16.935234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:16.935234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:16.935234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:16.935234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:16.935234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:16.935234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:16.935234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:16.935234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:16.935234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:16.935234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:16.935234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:16.935234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:16.936234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:16.936234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:16.936234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 21:50:16.936234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 21:50:16.936234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 21:50:16.936234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 21:50:16.936234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 21:50:16.936234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 21:50:16.936234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 21:50:16.936234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:16.939071 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 321/651] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:50:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:16.940628 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37348ae2-1ac2-4684-8104-c1cdadabe839 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:16.946791 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 331/652] 10.222.0.13 () {56 vars in 1137 bytes} [Tue Jun 7 21:50:16 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:16.948405 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2900d4-d856-4ed8-992d-5ec38de8576e None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:17.045580 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:17.070853 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37348ae2-1ac2-4684-8104-c1cdadabe839 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:17.075188 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2900d4-d856-4ed8-992d-5ec38de8576e None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:17.100989 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:17.117388 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7210525-a006-4c18-8e4e-103a7b7efca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:17.118019 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8285bda2-3b75-41ee-84da-e7e14cd2d81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:17.118669 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7210525-a006-4c18-8e4e-103a7b7efca7 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:17.118779 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8285bda2-3b75-41ee-84da-e7e14cd2d81f None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:17.118954 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8285bda2-3b75-41ee-84da-e7e14cd2d81f None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:17.119229 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8285bda2-3b75-41ee-84da-e7e14cd2d81f None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:17.119608 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7210525-a006-4c18-8e4e-103a7b7efca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:17.120096 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8285bda2-3b75-41ee-84da-e7e14cd2d81f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:17.120202 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7210525-a006-4c18-8e4e-103a7b7efca7 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:17.120398 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8285bda2-3b75-41ee-84da-e7e14cd2d81f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:17.121305 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7210525-a006-4c18-8e4e-103a7b7efca7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:17.121806 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7210525-a006-4c18-8e4e-103a7b7efca7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:17.121923 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8285bda2-3b75-41ee-84da-e7e14cd2d81f None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:17.123717 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7210525-a006-4c18-8e4e-103a7b7efca7 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:17.135000 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 322/653] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Jun 7 21:50:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:17.136097 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7210525-a006-4c18-8e4e-103a7b7efca7 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:17.144048 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 332/654] 10.222.0.13 () {56 vars in 1137 bytes} [Tue Jun 7 21:50:16 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:17.145208 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8285bda2-3b75-41ee-84da-e7e14cd2d81f None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:17.258854 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8285bda2-3b75-41ee-84da-e7e14cd2d81f None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:17.260708 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7210525-a006-4c18-8e4e-103a7b7efca7 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:17.295981 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ff24c3-7d8b-446d-bd9a-3c8807834ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:17.297496 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff24c3-7d8b-446d-bd9a-3c8807834ded None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:17.298036 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff24c3-7d8b-446d-bd9a-3c8807834ded None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:17.298359 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d2e8dd-a24f-4c7c-a596-d69dd7a93e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:17.298702 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff24c3-7d8b-446d-bd9a-3c8807834ded None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:17.299285 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2e8dd-a24f-4c7c-a596-d69dd7a93e11 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:17.299677 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2e8dd-a24f-4c7c-a596-d69dd7a93e11 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:17.299960 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2e8dd-a24f-4c7c-a596-d69dd7a93e11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:17.301694 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:17.310941 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ff24c3-7d8b-446d-bd9a-3c8807834ded None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:17.312660 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ff24c3-7d8b-446d-bd9a-3c8807834ded None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:17.332194 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-59ff24c3-7d8b-446d-bd9a-3c8807834ded None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:17.332194 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:17.332194 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:17.332194 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:17.332194 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:17.332194 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:17.332194 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:17.332194 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:17.332194 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:17.332194 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:17.332194 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:17.332194 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:17.332194 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:17.332194 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:17.332194 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:17.333552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:17.333552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:17.333552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:17.333552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:17.333552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:17.333552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:17.333552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:17.333552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:17.333552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:17.333552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:17.333552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:17.333552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:17.333552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:17.333552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:17.333552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:17.333552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:17.333552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:17.334457 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:17.334457 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:17.334457 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:17.334457 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:17.336431 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 333/655] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:17.337384 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ff24c3-7d8b-446d-bd9a-3c8807834ded None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:17.367508 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:17.383682 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d2e8dd-a24f-4c7c-a596-d69dd7a93e11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:17.385236 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d2e8dd-a24f-4c7c-a596-d69dd7a93e11 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:17.444993 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ff24c3-7d8b-446d-bd9a-3c8807834ded None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:17.481056 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b25acb-e1a3-4570-9498-f75bd0a29ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:17.482023 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b25acb-e1a3-4570-9498-f75bd0a29ad3 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:17.482528 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b25acb-e1a3-4570-9498-f75bd0a29ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:17.482988 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b25acb-e1a3-4570-9498-f75bd0a29ad3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:17.496550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 323/656] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:50:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 21:50:17.497606 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d2e8dd-a24f-4c7c-a596-d69dd7a93e11 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:17.545635 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:17.561919 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b25acb-e1a3-4570-9498-f75bd0a29ad3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:17.563734 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b25acb-e1a3-4570-9498-f75bd0a29ad3 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:17.594103 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:17.613023 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d2e8dd-a24f-4c7c-a596-d69dd7a93e11 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:17.649858 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d6c12f-a058-41e9-9774-8d43a7996b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:17.650672 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d6c12f-a058-41e9-9774-8d43a7996b0c None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:17.650816 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d6c12f-a058-41e9-9774-8d43a7996b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:17.651083 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d6c12f-a058-41e9-9774-8d43a7996b0c None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:17.651866 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d6c12f-a058-41e9-9774-8d43a7996b0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:17.652176 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d6c12f-a058-41e9-9774-8d43a7996b0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:17.653598 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d6c12f-a058-41e9-9774-8d43a7996b0c None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:17.671684 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 324/657] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:17.672681 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d6c12f-a058-41e9-9774-8d43a7996b0c None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:17.677083 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 334/658] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:50:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 21:50:17.678249 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b25acb-e1a3-4570-9498-f75bd0a29ad3 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:17.770281 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:17.785961 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d6c12f-a058-41e9-9774-8d43a7996b0c None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:17.787538 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b25acb-e1a3-4570-9498-f75bd0a29ad3 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:17.821899 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3475d-6be0-4930-8ed2-2ce956d4e1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:17.822617 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3475d-6be0-4930-8ed2-2ce956d4e1b7 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:17.822915 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3475d-6be0-4930-8ed2-2ce956d4e1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:17.823176 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3475d-6be0-4930-8ed2-2ce956d4e1b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:17.824926 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5798961f-ad41-442d-ac83-b36dd469a1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:17.825940 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5798961f-ad41-442d-ac83-b36dd469a1b8 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:17.826427 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5798961f-ad41-442d-ac83-b36dd469a1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:17.826922 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5798961f-ad41-442d-ac83-b36dd469a1b8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:17.834579 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5798961f-ad41-442d-ac83-b36dd469a1b8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:17.835698 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a3475d-6be0-4930-8ed2-2ce956d4e1b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:17.836573 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5798961f-ad41-442d-ac83-b36dd469a1b8 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:17.837168 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a3475d-6be0-4930-8ed2-2ce956d4e1b7 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:17.844494 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:17.855552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-5798961f-ad41-442d-ac83-b36dd469a1b8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 21:50:17.855552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:17.855552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:17.855552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:17.855552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:17.855552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:17.855552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:17.855552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:17.855552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:17.855552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:17.855552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:17.855552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:17.855552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 21:50:17.855552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 21:50:17.855552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 21:50:17.855552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 21:50:17.856537 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:17.856537 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:17.856537 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 21:50:17.856537 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:17.856537 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 21:50:17.856537 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:17.856537 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:17.856537 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:17.856537 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:17.856537 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:17.856537 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:17.856537 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:17.856537 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:17.856537 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:17.856537 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:17.856537 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:17.856537 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:17.856537 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:17.857502 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:17.857502 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:17.857502 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 21:50:17.857502 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 21:50:17.857502 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 21:50:17.857502 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 21:50:17.857502 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 21:50:17.857502 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 21:50:17.857502 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 21:50:17.857502 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:17.858533 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-45a3475d-6be0-4930-8ed2-2ce956d4e1b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:17.858533 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:17.858533 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:17.858533 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:17.858533 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:17.858533 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:17.858533 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:17.858533 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:17.858533 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:17.858533 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:17.858533 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:17.858533 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:17.858533 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:17.858533 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:17.858533 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:17.859401 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:17.859401 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:17.859401 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:17.859401 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:17.859401 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:17.859401 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:17.859401 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:17.859401 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:17.859401 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:17.859401 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:17.859401 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:17.859401 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:17.859401 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:17.859401 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:17.859401 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:17.859401 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:17.859401 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:17.861479 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:17.861479 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:17.861479 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:17.861479 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:17.861479 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 325/659] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:17.861479 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3475d-6be0-4930-8ed2-2ce956d4e1b7 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:17.864668 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 335/660] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jun 7 21:50:17 2022] GET /identity/v3/users/neutron => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:17.865862 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5798961f-ad41-442d-ac83-b36dd469a1b8 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:17.974639 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5798961f-ad41-442d-ac83-b36dd469a1b8 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:17.999257 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3475d-6be0-4930-8ed2-2ce956d4e1b7 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:18.014985 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7f3e0a-b767-4b23-8754-87dc32e4eb88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:18.015690 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7f3e0a-b767-4b23-8754-87dc32e4eb88 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:18.015977 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7f3e0a-b767-4b23-8754-87dc32e4eb88 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:18.016258 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7f3e0a-b767-4b23-8754-87dc32e4eb88 None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:18.016983 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7f3e0a-b767-4b23-8754-87dc32e4eb88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:18.017358 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7f3e0a-b767-4b23-8754-87dc32e4eb88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:18.018892 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7f3e0a-b767-4b23-8754-87dc32e4eb88 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:18.035577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 336/661] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:50:18.035577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7f3e0a-b767-4b23-8754-87dc32e4eb88 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:18.037477 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed8a40-9d87-4a5e-8f13-04a2b1eaa00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:18.038115 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed8a40-9d87-4a5e-8f13-04a2b1eaa00b None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:18.038402 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed8a40-9d87-4a5e-8f13-04a2b1eaa00b None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:18.038681 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed8a40-9d87-4a5e-8f13-04a2b1eaa00b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:18.043233 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:18.050230 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aed8a40-9d87-4a5e-8f13-04a2b1eaa00b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:18.051716 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aed8a40-9d87-4a5e-8f13-04a2b1eaa00b None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:18.069117 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-4aed8a40-9d87-4a5e-8f13-04a2b1eaa00b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:18.069117 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:18.069117 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:18.069117 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:18.069117 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:18.069117 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:18.069117 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:18.069117 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:18.069117 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:18.069117 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:18.069117 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:18.069117 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:18.069117 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:18.069117 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:18.069117 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:18.070106 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:18.070106 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:18.070106 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:18.070106 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:18.070106 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:18.070106 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:18.070106 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:18.070106 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:18.070106 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:18.070106 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:18.070106 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:18.070106 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:18.070106 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:18.070106 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:18.070106 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:18.070106 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:18.070106 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:18.071004 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:18.071004 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:18.071004 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:18.071004 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:18.074901 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 326/662] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:18.075979 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894dc678-f30e-4e2d-9ef7-9c28ae68ff71 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:18.076268 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894dc678-f30e-4e2d-9ef7-9c28ae68ff71 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:18.076549 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894dc678-f30e-4e2d-9ef7-9c28ae68ff71 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:18.082147 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 327/663] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:18.082147 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894dc678-f30e-4e2d-9ef7-9c28ae68ff71 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:18.132048 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:18.142167 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7f3e0a-b767-4b23-8754-87dc32e4eb88 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:18.176010 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b905d8-e877-47a8-98dc-79198387ccae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:18.176816 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b905d8-e877-47a8-98dc-79198387ccae None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:18.177172 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b905d8-e877-47a8-98dc-79198387ccae None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:18.177511 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b905d8-e877-47a8-98dc-79198387ccae None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:18.178367 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b905d8-e877-47a8-98dc-79198387ccae None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:18.178682 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b905d8-e877-47a8-98dc-79198387ccae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:18.180137 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b905d8-e877-47a8-98dc-79198387ccae None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:18.188096 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894dc678-f30e-4e2d-9ef7-9c28ae68ff71 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:18.202802 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 337/664] 10.222.0.13 () {56 vars in 1139 bytes} [Tue Jun 7 21:50:18 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:18.202802 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b905d8-e877-47a8-98dc-79198387ccae None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:18.229034 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7acf38-b6dc-4cc5-b65e-9b38393475b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:18.230111 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7acf38-b6dc-4cc5-b65e-9b38393475b2 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:18.230383 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7acf38-b6dc-4cc5-b65e-9b38393475b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:18.230662 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7acf38-b6dc-4cc5-b65e-9b38393475b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:18.237855 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7acf38-b6dc-4cc5-b65e-9b38393475b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:18.239206 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7acf38-b6dc-4cc5-b65e-9b38393475b2 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:18.254569 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-6d7acf38-b6dc-4cc5-b65e-9b38393475b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:18.254569 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:18.254569 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:18.254569 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:18.254569 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:18.254569 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:18.254569 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:18.254569 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:18.254569 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:18.254569 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:18.254569 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:18.254569 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:18.254569 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 21:50:18.254569 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 21:50:18.254569 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 21:50:18.255554 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 21:50:18.255554 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:18.255554 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:18.255554 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:18.255554 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:18.255554 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:18.255554 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:18.255554 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:18.255554 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:18.255554 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:18.255554 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:18.255554 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:18.255554 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:18.255554 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:18.255554 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:18.255554 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:18.255554 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:18.256650 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 21:50:18.256650 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 21:50:18.256650 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 21:50:18.256650 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 21:50:18.256650 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 21:50:18.256650 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 21:50:18.256650 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:18.256650 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:18.260367 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 328/665] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:50:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:18.261634 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff1cef-4cac-40cb-b2ff-09db901435f6 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:18.261926 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff1cef-4cac-40cb-b2ff-09db901435f6 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:18.262210 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff1cef-4cac-40cb-b2ff-09db901435f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:18.272894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:18.280888 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-36ff1cef-4cac-40cb-b2ff-09db901435f6 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:18.282133 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-36ff1cef-4cac-40cb-b2ff-09db901435f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:18.312107 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b905d8-e877-47a8-98dc-79198387ccae None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:18.350249 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00316cb-17f9-4458-aa5b-67e4215b8464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:18.350934 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00316cb-17f9-4458-aa5b-67e4215b8464 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:18.351114 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00316cb-17f9-4458-aa5b-67e4215b8464 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:18.351417 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00316cb-17f9-4458-aa5b-67e4215b8464 None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:18.352362 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00316cb-17f9-4458-aa5b-67e4215b8464 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:18.352471 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00316cb-17f9-4458-aa5b-67e4215b8464 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:18.353846 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00316cb-17f9-4458-aa5b-67e4215b8464 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:18.367392 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 329/666] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:50:18.367392 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ff1cef-4cac-40cb-b2ff-09db901435f6 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:18.373823 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 338/667] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:18.374757 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00316cb-17f9-4458-aa5b-67e4215b8464 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:18.410137 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:18.472253 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ff1cef-4cac-40cb-b2ff-09db901435f6 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:18.481938 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00316cb-17f9-4458-aa5b-67e4215b8464 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:18.506443 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:18.511314 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf8551d-ec32-44b0-8ddb-311ed3565522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:18.511995 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8551d-ec32-44b0-8ddb-311ed3565522 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:18.512277 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8551d-ec32-44b0-8ddb-311ed3565522 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:18.512552 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8551d-ec32-44b0-8ddb-311ed3565522 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:18.519751 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e1a82-a2e2-4a49-a86f-1703e2ea4b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:18.520424 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e1a82-a2e2-4a49-a86f-1703e2ea4b76 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:18.520715 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e1a82-a2e2-4a49-a86f-1703e2ea4b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:18.520996 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e1a82-a2e2-4a49-a86f-1703e2ea4b76 None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:18.521792 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e1a82-a2e2-4a49-a86f-1703e2ea4b76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:18.522106 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e1a82-a2e2-4a49-a86f-1703e2ea4b76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:18.523586 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e1a82-a2e2-4a49-a86f-1703e2ea4b76 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:18.546921 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 339/668] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jun 7 21:50:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:18.546921 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e1a82-a2e2-4a49-a86f-1703e2ea4b76 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:18.593047 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf8551d-ec32-44b0-8ddb-311ed3565522 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:18.594596 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf8551d-ec32-44b0-8ddb-311ed3565522 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:18.643310 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:18.660913 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e1a82-a2e2-4a49-a86f-1703e2ea4b76 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:18.694994 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101ed88-6462-496d-b641-8e7169490326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:18.695812 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101ed88-6462-496d-b641-8e7169490326 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:18.696092 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101ed88-6462-496d-b641-8e7169490326 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:18.696366 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101ed88-6462-496d-b641-8e7169490326 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:18.702832 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4101ed88-6462-496d-b641-8e7169490326 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:18.704273 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4101ed88-6462-496d-b641-8e7169490326 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:18.707017 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 330/669] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:50:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 21:50:18.707885 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf8551d-ec32-44b0-8ddb-311ed3565522 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:18.718583 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-4101ed88-6462-496d-b641-8e7169490326 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:18.718583 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:18.718583 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:18.718583 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:18.718583 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:18.718583 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:18.718583 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:18.718583 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:18.718583 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:18.718583 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:18.718583 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:18.718583 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:18.718583 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 21:50:18.718583 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 21:50:18.718583 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 21:50:18.719543 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 21:50:18.719543 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:18.719543 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:18.719543 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:18.719543 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:18.719543 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:18.719543 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:18.719543 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:18.719543 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:18.719543 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:18.719543 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:18.719543 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:18.719543 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:18.719543 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:18.719543 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:18.719543 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:18.719543 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:18.720424 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 21:50:18.720424 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 21:50:18.720424 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 21:50:18.720424 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 21:50:18.720424 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 21:50:18.720424 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 21:50:18.720424 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:18.720424 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:18.720424 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 340/670] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:50:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:18.721591 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101ed88-6462-496d-b641-8e7169490326 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:18.769892 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:18.814788 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf8551d-ec32-44b0-8ddb-311ed3565522 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:18.823862 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101ed88-6462-496d-b641-8e7169490326 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:18.850261 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799443d6-7a23-4f11-94f7-1122c1d661cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:18.851055 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799443d6-7a23-4f11-94f7-1122c1d661cc None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:18.851351 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799443d6-7a23-4f11-94f7-1122c1d661cc None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:18.851644 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799443d6-7a23-4f11-94f7-1122c1d661cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:18.859621 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:18.863186 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799443d6-7a23-4f11-94f7-1122c1d661cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:50:18.865440 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95050a11-1dc8-403c-bdcc-85aefa4262a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:18.866209 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95050a11-1dc8-403c-bdcc-85aefa4262a3 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:18.866391 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95050a11-1dc8-403c-bdcc-85aefa4262a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:18.866661 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95050a11-1dc8-403c-bdcc-85aefa4262a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:18.867519 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95050a11-1dc8-403c-bdcc-85aefa4262a3 None admin] RBAC: Adding query filter params (role.id=2c7f73c184d0406bbf2899465953342b, scope.project.id=0219dbda54274fdc9bdccbd960c30c6b, user.id=96987b04471944bc86ab44ff9936c732) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:18.867828 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95050a11-1dc8-403c-bdcc-85aefa4262a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:18.869258 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95050a11-1dc8-403c-bdcc-85aefa4262a3 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:18.871294 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-799443d6-7a23-4f11-94f7-1122c1d661cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:50:18.871294 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:18.871294 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:18.871294 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:18.871294 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:18.871294 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:18.871294 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:18.871294 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:18.871294 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:18.871294 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:18.871294 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:18.871294 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:18.871294 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:50:18.871294 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:50:18.871294 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:50:18.871294 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:50:18.872237 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:50:18.872237 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:50:18.872237 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:50:18.872237 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:50:18.872237 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:18.872237 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:18.872237 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:18.872237 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:18.872237 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:18.872237 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:18.872237 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:18.872237 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:18.872237 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:18.872237 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:18.872237 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:18.872237 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:18.873142 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:18.873142 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:18.873142 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:18.873142 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:18.873142 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:50:18.873142 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:50:18.873142 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:50:18.873142 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:50:18.873142 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:50:18.873142 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:50:18.873142 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:50:18.873142 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:18.873142 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 331/671] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jun 7 21:50:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:18.873919 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60041f53-445d-480d-8ca3-4350a0f0face None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:18.874124 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60041f53-445d-480d-8ca3-4350a0f0face None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:18.874370 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60041f53-445d-480d-8ca3-4350a0f0face None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:18.875873 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 332/672] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:18.876521 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60041f53-445d-480d-8ca3-4350a0f0face None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:18.881453 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 341/673] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:50:18 2022] GET /identity/v3/role_assignments?user.id=96987b04471944bc86ab44ff9936c732&scope.project.id=0219dbda54274fdc9bdccbd960c30c6b&role.id=2c7f73c184d0406bbf2899465953342b => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:18.882559 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95050a11-1dc8-403c-bdcc-85aefa4262a3 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:19.000724 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95050a11-1dc8-403c-bdcc-85aefa4262a3 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:19.006145 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60041f53-445d-480d-8ca3-4350a0f0face None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:19.011589 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:19.040144 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31ec063-7ff2-4dc4-baab-bbd15c307bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:19.040935 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31ec063-7ff2-4dc4-baab-bbd15c307bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:19.041343 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31ec063-7ff2-4dc4-baab-bbd15c307bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:19.041624 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31ec063-7ff2-4dc4-baab-bbd15c307bb1 None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:19.042488 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31ec063-7ff2-4dc4-baab-bbd15c307bb1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:19.042850 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31ec063-7ff2-4dc4-baab-bbd15c307bb1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:19.044407 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31ec063-7ff2-4dc4-baab-bbd15c307bb1 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:19.046110 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8d29cb-8388-480e-ab1b-1264e0736d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:19.047019 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8d29cb-8388-480e-ab1b-1264e0736d34 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:19.047310 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8d29cb-8388-480e-ab1b-1264e0736d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:19.047620 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8d29cb-8388-480e-ab1b-1264e0736d34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:19.058770 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 342/674] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jun 7 21:50:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:19.059927 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8d29cb-8388-480e-ab1b-1264e0736d34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:19.060238 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31ec063-7ff2-4dc4-baab-bbd15c307bb1 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:19.061417 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8d29cb-8388-480e-ab1b-1264e0736d34 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:19.084204 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-fb8d29cb-8388-480e-ab1b-1264e0736d34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:19.084204 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:19.084204 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:19.084204 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:19.084204 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:19.084204 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:19.084204 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:19.084204 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:19.084204 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:19.084204 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:19.084204 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:19.084204 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:19.084204 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:19.084204 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:19.084204 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:19.085162 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:19.085162 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:19.085162 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:19.085162 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:19.085162 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:19.085162 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:19.085162 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:19.085162 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:19.085162 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:19.085162 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:19.085162 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:19.085162 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:19.085162 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:19.085162 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:19.085162 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:19.085162 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:19.085162 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:19.086052 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:19.086052 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:19.086052 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:19.086052 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:19.086052 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 333/675] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:19.086052 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e834b82-b7c6-4c96-9ef5-7fb686f75612 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:19.086430 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e834b82-b7c6-4c96-9ef5-7fb686f75612 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:19.086430 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e834b82-b7c6-4c96-9ef5-7fb686f75612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:19.105262 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-8e834b82-b7c6-4c96-9ef5-7fb686f75612 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:19.106470 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-8e834b82-b7c6-4c96-9ef5-7fb686f75612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:19.161381 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:19.180626 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31ec063-7ff2-4dc4-baab-bbd15c307bb1 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:19.182962 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 334/676] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:50:19.183846 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e834b82-b7c6-4c96-9ef5-7fb686f75612 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:19.225487 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d42aa5-59ad-4078-a728-b8af6802ebeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:19.226314 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d42aa5-59ad-4078-a728-b8af6802ebeb None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:19.226698 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d42aa5-59ad-4078-a728-b8af6802ebeb None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:19.227102 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d42aa5-59ad-4078-a728-b8af6802ebeb None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:19.227986 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d42aa5-59ad-4078-a728-b8af6802ebeb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:19.228380 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d42aa5-59ad-4078-a728-b8af6802ebeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:19.229826 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d42aa5-59ad-4078-a728-b8af6802ebeb None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:19.242940 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 343/677] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:50:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:19.242940 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d42aa5-59ad-4078-a728-b8af6802ebeb None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:19.271135 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:19.297283 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e834b82-b7c6-4c96-9ef5-7fb686f75612 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:19.332561 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc52ae-d871-4a9f-90f2-7d51d3d6e820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:19.333214 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc52ae-d871-4a9f-90f2-7d51d3d6e820 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:19.333492 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc52ae-d871-4a9f-90f2-7d51d3d6e820 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:19.333763 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc52ae-d871-4a9f-90f2-7d51d3d6e820 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:19.334591 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbc52ae-d871-4a9f-90f2-7d51d3d6e820 None admin] RBAC: Adding query filter params (role.id=2c7f73c184d0406bbf2899465953342b, scope.project.id=0219dbda54274fdc9bdccbd960c30c6b, user.id=ea8a1e2cf1da4f87b8d4e16f525fc759) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:19.334905 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbc52ae-d871-4a9f-90f2-7d51d3d6e820 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:19.336351 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbc52ae-d871-4a9f-90f2-7d51d3d6e820 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:19.351372 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 335/678] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:50:19 2022] GET /identity/v3/role_assignments?user.id=ea8a1e2cf1da4f87b8d4e16f525fc759&scope.project.id=0219dbda54274fdc9bdccbd960c30c6b&role.id=2c7f73c184d0406bbf2899465953342b => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:19.351372 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc52ae-d871-4a9f-90f2-7d51d3d6e820 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:19.358084 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d42aa5-59ad-4078-a728-b8af6802ebeb None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:19.405258 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:19.406009 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c978dd-a85a-4233-8256-10388f68d2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:19.407089 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c978dd-a85a-4233-8256-10388f68d2c5 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:19.407388 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c978dd-a85a-4233-8256-10388f68d2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:19.407669 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c978dd-a85a-4233-8256-10388f68d2c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:19.409182 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c978dd-a85a-4233-8256-10388f68d2c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:19.409484 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c978dd-a85a-4233-8256-10388f68d2c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:19.412179 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c978dd-a85a-4233-8256-10388f68d2c5 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:19.428942 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 344/679] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:19.429950 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c978dd-a85a-4233-8256-10388f68d2c5 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:19.472905 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc52ae-d871-4a9f-90f2-7d51d3d6e820 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:19.514062 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a5ea7-1029-4029-ad6f-f53a5463e80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:19.515009 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a5ea7-1029-4029-ad6f-f53a5463e80b None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:19.515377 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a5ea7-1029-4029-ad6f-f53a5463e80b None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:19.515722 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a5ea7-1029-4029-ad6f-f53a5463e80b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:19.528785 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623a5ea7-1029-4029-ad6f-f53a5463e80b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:50:19.538244 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-623a5ea7-1029-4029-ad6f-f53a5463e80b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 21:50:19.538244 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:19.538244 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:19.538244 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:19.538244 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:19.538244 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:19.538244 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:19.538244 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:19.538244 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:19.538244 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:19.538244 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:19.538244 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:19.538244 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:50:19.538244 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:50:19.538244 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:50:19.538244 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:50:19.539293 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:50:19.539293 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:50:19.539293 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:50:19.539293 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:50:19.539293 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:19.539293 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:19.539293 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:19.539293 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:19.539293 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:19.539293 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:19.539293 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:19.539293 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:19.539293 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:19.539293 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:19.539293 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:19.539293 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:19.540180 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:19.540180 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:19.540180 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:19.540180 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:19.540180 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:50:19.540180 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:50:19.540180 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:50:19.540180 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:50:19.540180 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:50:19.540180 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:50:19.540180 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 21:50:19.540180 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:19.544393 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 336/680] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jun 7 21:50:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:19.545352 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a5ea7-1029-4029-ad6f-f53a5463e80b None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:19.569506 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:19.581798 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c978dd-a85a-4233-8256-10388f68d2c5 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:19.629442 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e9879d-b46e-4adc-908d-c49d33d9e29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:19.630601 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9879d-b46e-4adc-908d-c49d33d9e29c None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:19.630883 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9879d-b46e-4adc-908d-c49d33d9e29c None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:19.631159 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9879d-b46e-4adc-908d-c49d33d9e29c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:19.646037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e9879d-b46e-4adc-908d-c49d33d9e29c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:19.647584 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e9879d-b46e-4adc-908d-c49d33d9e29c None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:19.673800 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-20e9879d-b46e-4adc-908d-c49d33d9e29c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:19.673800 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:19.673800 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:19.673800 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:19.673800 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:19.673800 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:19.673800 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:19.673800 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:19.673800 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:19.673800 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:19.673800 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:19.673800 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:19.673800 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:19.673800 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:19.673800 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:19.675514 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:19.675514 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:19.675514 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:19.675514 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:19.675514 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:19.675514 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:19.675514 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:19.675514 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:19.675514 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:19.675514 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:19.675514 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:19.675514 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:19.675514 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:19.675514 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:19.675514 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:19.675514 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:19.675514 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:19.676627 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:19.676627 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:19.676627 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:19.676627 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:19.677640 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a5ea7-1029-4029-ad6f-f53a5463e80b None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:19.683279 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 345/681] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:19.683279 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e9879d-b46e-4adc-908d-c49d33d9e29c None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:19.692344 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:19.722765 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517e59eb-a68e-4a31-98ba-6a291a9698ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:19.723590 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e59eb-a68e-4a31-98ba-6a291a9698ff None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:19.723883 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e59eb-a68e-4a31-98ba-6a291a9698ff None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:19.724144 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e59eb-a68e-4a31-98ba-6a291a9698ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:19.731430 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517e59eb-a68e-4a31-98ba-6a291a9698ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:19.732862 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517e59eb-a68e-4a31-98ba-6a291a9698ff None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:19.749050 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-517e59eb-a68e-4a31-98ba-6a291a9698ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:19.749050 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:19.749050 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:19.749050 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:19.749050 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:19.749050 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:19.749050 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:19.749050 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:19.749050 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:19.749050 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:19.749050 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:19.749050 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:19.749050 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 21:50:19.749050 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 21:50:19.749050 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 21:50:19.749999 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 21:50:19.749999 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:19.749999 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:19.749999 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:19.749999 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:19.749999 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:19.749999 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:19.749999 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:19.749999 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:19.749999 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:19.749999 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:19.749999 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:19.749999 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:19.749999 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:19.749999 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:19.749999 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:19.749999 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:19.750990 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 21:50:19.750990 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 21:50:19.750990 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 21:50:19.750990 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 21:50:19.750990 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 21:50:19.750990 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 21:50:19.750990 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:19.750990 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:19.750990 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 337/682] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:50:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:19.752114 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517e59eb-a68e-4a31-98ba-6a291a9698ff None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:19.815300 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e9879d-b46e-4adc-908d-c49d33d9e29c None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:19.851547 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c05f981-0c09-4a23-8855-d847932e55d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:19.852585 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c05f981-0c09-4a23-8855-d847932e55d9 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:19.853104 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c05f981-0c09-4a23-8855-d847932e55d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:19.853592 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c05f981-0c09-4a23-8855-d847932e55d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:19.854536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c05f981-0c09-4a23-8855-d847932e55d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:19.855044 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c05f981-0c09-4a23-8855-d847932e55d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:19.856650 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c05f981-0c09-4a23-8855-d847932e55d9 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:19.865542 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 346/683] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Jun 7 21:50:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:19.866555 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c05f981-0c09-4a23-8855-d847932e55d9 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:19.870147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:19.876435 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517e59eb-a68e-4a31-98ba-6a291a9698ff None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:19.916097 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce221e0-1d18-43ec-8344-420b23b5868e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:19.916799 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce221e0-1d18-43ec-8344-420b23b5868e None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:19.917080 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce221e0-1d18-43ec-8344-420b23b5868e None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:19.917357 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce221e0-1d18-43ec-8344-420b23b5868e None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:19.918089 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce221e0-1d18-43ec-8344-420b23b5868e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:19.918379 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce221e0-1d18-43ec-8344-420b23b5868e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:19.919832 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce221e0-1d18-43ec-8344-420b23b5868e None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:19.936677 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 338/684] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:19.936677 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce221e0-1d18-43ec-8344-420b23b5868e None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:19.969206 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:19.991592 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c05f981-0c09-4a23-8855-d847932e55d9 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:20.034808 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45024f81-8de4-429c-855c-ce0214de8b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:20.035796 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45024f81-8de4-429c-855c-ce0214de8b71 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:20.036720 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45024f81-8de4-429c-855c-ce0214de8b71 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:20.037162 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45024f81-8de4-429c-855c-ce0214de8b71 None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:20.038102 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45024f81-8de4-429c-855c-ce0214de8b71 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:20.039008 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45024f81-8de4-429c-855c-ce0214de8b71 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:20.041337 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45024f81-8de4-429c-855c-ce0214de8b71 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:20.059118 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 347/685] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jun 7 21:50:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:20.060449 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45024f81-8de4-429c-855c-ce0214de8b71 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:20.066102 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce221e0-1d18-43ec-8344-420b23b5868e None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:20.109805 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44efd445-0cd9-48fd-be0b-38d590406bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:20.110843 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44efd445-0cd9-48fd-be0b-38d590406bfb None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:20.111226 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44efd445-0cd9-48fd-be0b-38d590406bfb None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:20.112151 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44efd445-0cd9-48fd-be0b-38d590406bfb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:20.127571 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44efd445-0cd9-48fd-be0b-38d590406bfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:20.129467 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44efd445-0cd9-48fd-be0b-38d590406bfb None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:20.145649 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:20.150373 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-44efd445-0cd9-48fd-be0b-38d590406bfb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:20.150373 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:20.150373 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:20.150373 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:20.150373 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:20.150373 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:20.150373 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:20.150373 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:20.150373 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:20.150373 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:20.150373 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:20.150373 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:20.150373 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:20.150373 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:20.150373 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:20.151465 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:20.151465 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:20.151465 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:20.151465 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:20.151465 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:20.151465 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:20.151465 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:20.151465 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:20.151465 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:20.151465 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:20.151465 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:20.151465 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:20.151465 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:20.151465 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:20.151465 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:20.151465 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:20.151465 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:20.152579 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:20.152579 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:20.152579 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:20.152579 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:20.158843 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 339/686] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:20.159891 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44efd445-0cd9-48fd-be0b-38d590406bfb None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:20.200187 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45024f81-8de4-429c-855c-ce0214de8b71 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:20.244518 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:20.254183 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50691c82-fa5b-47c7-84eb-fcc984f4b14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:20.255196 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50691c82-fa5b-47c7-84eb-fcc984f4b14f None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:20.255668 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50691c82-fa5b-47c7-84eb-fcc984f4b14f None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:20.256115 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50691c82-fa5b-47c7-84eb-fcc984f4b14f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:20.267320 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50691c82-fa5b-47c7-84eb-fcc984f4b14f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:20.269878 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50691c82-fa5b-47c7-84eb-fcc984f4b14f None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:20.270476 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44efd445-0cd9-48fd-be0b-38d590406bfb None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:20.291907 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-50691c82-fa5b-47c7-84eb-fcc984f4b14f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 21:50:20.291907 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:20.291907 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:20.291907 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:20.291907 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:20.291907 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:20.291907 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:20.291907 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:20.291907 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:20.291907 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:20.291907 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:20.291907 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:20.291907 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 21:50:20.291907 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 21:50:20.291907 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 21:50:20.291907 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 21:50:20.293133 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:20.293133 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:20.293133 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 21:50:20.293133 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:20.293133 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 21:50:20.293133 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:20.293133 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:20.293133 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:20.293133 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:20.293133 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:20.293133 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:20.293133 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:20.293133 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:20.293133 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:20.293133 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:20.293133 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:20.293133 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:20.293133 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:20.294327 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:20.294327 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:20.294327 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 21:50:20.294327 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 21:50:20.294327 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 21:50:20.294327 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 21:50:20.294327 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 21:50:20.294327 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 21:50:20.294327 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 21:50:20.294327 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:20.302176 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 348/687] 10.222.0.13 () {56 vars in 1093 bytes} [Tue Jun 7 21:50:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:20.302176 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50691c82-fa5b-47c7-84eb-fcc984f4b14f None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:20.321297 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e118d3ae-d695-4bf2-99d7-736dff4a33cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:20.322171 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e118d3ae-d695-4bf2-99d7-736dff4a33cb None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:20.322620 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e118d3ae-d695-4bf2-99d7-736dff4a33cb None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:20.323051 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e118d3ae-d695-4bf2-99d7-736dff4a33cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:20.324660 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e118d3ae-d695-4bf2-99d7-736dff4a33cb None admin] RBAC: Adding query filter params (role.id=2c7f73c184d0406bbf2899465953342b, scope.project.id=0219dbda54274fdc9bdccbd960c30c6b, user.id=10242251e6284217bfb79c67b1567f75) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:20.325127 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e118d3ae-d695-4bf2-99d7-736dff4a33cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:20.327244 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e118d3ae-d695-4bf2-99d7-736dff4a33cb None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:20.336768 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 340/688] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:50:20 2022] GET /identity/v3/role_assignments?user.id=10242251e6284217bfb79c67b1567f75&scope.project.id=0219dbda54274fdc9bdccbd960c30c6b&role.id=2c7f73c184d0406bbf2899465953342b => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:20.337976 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e118d3ae-d695-4bf2-99d7-736dff4a33cb None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:20.493314 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50691c82-fa5b-47c7-84eb-fcc984f4b14f None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:20.520380 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e118d3ae-d695-4bf2-99d7-736dff4a33cb None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:20.559926 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8597eff0-cbeb-4705-b356-6d4435baf747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:20.560757 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8597eff0-cbeb-4705-b356-6d4435baf747 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:20.561145 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8597eff0-cbeb-4705-b356-6d4435baf747 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:20.561524 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8597eff0-cbeb-4705-b356-6d4435baf747 None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:20.562471 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8597eff0-cbeb-4705-b356-6d4435baf747 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:20.562905 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8597eff0-cbeb-4705-b356-6d4435baf747 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:20.565541 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8597eff0-cbeb-4705-b356-6d4435baf747 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:20.580729 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:20.583795 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:20.585840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e56d22-f060-4cf5-add7-386dfd76edb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:20.586519 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e56d22-f060-4cf5-add7-386dfd76edb2 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:20.586803 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e56d22-f060-4cf5-add7-386dfd76edb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:20.587077 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e56d22-f060-4cf5-add7-386dfd76edb2 None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:20.588093 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e56d22-f060-4cf5-add7-386dfd76edb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:20.588407 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e56d22-f060-4cf5-add7-386dfd76edb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:20.589482 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 341/689] 10.222.0.13 () {56 vars in 1133 bytes} [Tue Jun 7 21:50:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:20.589986 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e56d22-f060-4cf5-add7-386dfd76edb2 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:20.595164 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8597eff0-cbeb-4705-b356-6d4435baf747 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:20.608475 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 349/690] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:20.609854 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e56d22-f060-4cf5-add7-386dfd76edb2 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:20.723256 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e56d22-f060-4cf5-add7-386dfd76edb2 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:20.739009 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8597eff0-cbeb-4705-b356-6d4435baf747 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:20.777965 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c32996-3816-4c1c-880a-6d4f234da58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:20.778781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c32996-3816-4c1c-880a-6d4f234da58c None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:20.779068 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c32996-3816-4c1c-880a-6d4f234da58c None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:20.779368 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c32996-3816-4c1c-880a-6d4f234da58c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:20.791047 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c32996-3816-4c1c-880a-6d4f234da58c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:20.792573 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c32996-3816-4c1c-880a-6d4f234da58c None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:20.795660 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c8f8a7-421a-4b03-8946-7f1e6a2d0eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:20.796323 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8f8a7-421a-4b03-8946-7f1e6a2d0eb4 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:20.796608 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8f8a7-421a-4b03-8946-7f1e6a2d0eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:20.796886 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8f8a7-421a-4b03-8946-7f1e6a2d0eb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:20.810390 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-33c32996-3816-4c1c-880a-6d4f234da58c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 21:50:20.810390 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:20.810390 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:20.810390 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:20.810390 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:20.810390 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:20.810390 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:20.810390 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:20.810390 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:20.810390 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:20.810390 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:20.810390 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:20.810390 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 21:50:20.810390 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 21:50:20.810390 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 21:50:20.810390 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 21:50:20.811515 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:20.811515 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:20.811515 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 21:50:20.811515 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:20.811515 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 21:50:20.811515 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:20.811515 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:20.811515 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:20.811515 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:20.811515 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:20.811515 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:20.811515 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:20.811515 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:20.811515 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:20.811515 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:20.811515 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:20.811515 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:20.811515 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:20.812449 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:20.812449 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:20.812449 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 21:50:20.812449 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 21:50:20.812449 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 21:50:20.812449 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 21:50:20.812449 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 21:50:20.812449 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 21:50:20.812449 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 21:50:20.812449 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:20.815468 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 350/691] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jun 7 21:50:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:20.817986 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c32996-3816-4c1c-880a-6d4f234da58c None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:20.876174 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:20.880047 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c8f8a7-421a-4b03-8946-7f1e6a2d0eb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:20.882299 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c8f8a7-421a-4b03-8946-7f1e6a2d0eb4 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:20.902309 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:21.009110 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 342/692] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:50:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 21:50:21.015831 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c32996-3816-4c1c-880a-6d4f234da58c None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:21.017196 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c8f8a7-421a-4b03-8946-7f1e6a2d0eb4 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:21.058394 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb64714-a588-4f1a-bbb4-22f258cfed9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:21.059080 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb64714-a588-4f1a-bbb4-22f258cfed9e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:21.059383 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb64714-a588-4f1a-bbb4-22f258cfed9e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:21.059667 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb64714-a588-4f1a-bbb4-22f258cfed9e None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:21.060516 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb64714-a588-4f1a-bbb4-22f258cfed9e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:21.060828 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb64714-a588-4f1a-bbb4-22f258cfed9e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:21.062260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb64714-a588-4f1a-bbb4-22f258cfed9e None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:21.093182 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 351/693] 10.222.0.13 () {56 vars in 1135 bytes} [Tue Jun 7 21:50:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:50:21.104026 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb64714-a588-4f1a-bbb4-22f258cfed9e None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:21.217804 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:21.224677 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c8f8a7-421a-4b03-8946-7f1e6a2d0eb4 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:21.273925 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb64714-a588-4f1a-bbb4-22f258cfed9e None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:21.275531 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:21.290614 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eca825-fc03-4c49-985c-7bfa4cb66cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:21.291625 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eca825-fc03-4c49-985c-7bfa4cb66cea None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:21.291909 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eca825-fc03-4c49-985c-7bfa4cb66cea None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:21.292186 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eca825-fc03-4c49-985c-7bfa4cb66cea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:21.306657 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eca825-fc03-4c49-985c-7bfa4cb66cea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:21.308146 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eca825-fc03-4c49-985c-7bfa4cb66cea None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:21.312140 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b037b08-3bdf-4df9-a483-e9f178c99bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:21.312788 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b037b08-3bdf-4df9-a483-e9f178c99bfb None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:21.313066 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b037b08-3bdf-4df9-a483-e9f178c99bfb None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:21.313342 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b037b08-3bdf-4df9-a483-e9f178c99bfb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:21.327815 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-86eca825-fc03-4c49-985c-7bfa4cb66cea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:21.327815 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:21.327815 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:21.327815 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:21.327815 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:21.327815 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:21.327815 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:21.327815 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:21.327815 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:21.327815 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:21.327815 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:21.327815 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:21.327815 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:21.327815 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:21.327815 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:21.328768 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:21.328768 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:21.328768 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:21.328768 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:21.328768 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:21.328768 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:21.328768 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:21.328768 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:21.328768 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:21.328768 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:21.328768 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:21.328768 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:21.328768 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:21.328768 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:21.328768 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:21.328768 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:21.328768 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:21.329690 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:21.329690 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:21.329690 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:21.329690 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:21.329690 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 343/694] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:21.331479 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d02842-de36-4de4-926f-fe2e229a6c1d None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:21.331587 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d02842-de36-4de4-926f-fe2e229a6c1d None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:21.335233 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d02842-de36-4de4-926f-fe2e229a6c1d None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:21.335233 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 344/695] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:21.335233 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d02842-de36-4de4-926f-fe2e229a6c1d None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:21.394284 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b037b08-3bdf-4df9-a483-e9f178c99bfb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:21.395832 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b037b08-3bdf-4df9-a483-e9f178c99bfb None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:21.448471 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d02842-de36-4de4-926f-fe2e229a6c1d None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:21.486790 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e14f3b-f5f2-4e83-96b0-d6a47ffa8822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:21.487505 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e14f3b-f5f2-4e83-96b0-d6a47ffa8822 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:21.487794 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e14f3b-f5f2-4e83-96b0-d6a47ffa8822 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:21.488171 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e14f3b-f5f2-4e83-96b0-d6a47ffa8822 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:21.488912 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e14f3b-f5f2-4e83-96b0-d6a47ffa8822 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:21.489222 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e14f3b-f5f2-4e83-96b0-d6a47ffa8822 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:21.490598 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e14f3b-f5f2-4e83-96b0-d6a47ffa8822 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:21.504746 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 345/696] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:21.505787 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 352/697] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:50:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 21:50:21.509940 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b037b08-3bdf-4df9-a483-e9f178c99bfb None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:21.510766 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae14f38-4177-46b1-9c64-adffd472229a None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:21.511071 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae14f38-4177-46b1-9c64-adffd472229a None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:21.511372 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae14f38-4177-46b1-9c64-adffd472229a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:21.513031 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:21.533201 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-3ae14f38-4177-46b1-9c64-adffd472229a None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:21.535286 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-3ae14f38-4177-46b1-9c64-adffd472229a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:21.559700 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:21.613381 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 346/698] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:50:21.614333 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae14f38-4177-46b1-9c64-adffd472229a None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:21.618118 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b037b08-3bdf-4df9-a483-e9f178c99bfb None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:21.665221 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8238b664-1eab-4926-a76a-ed6aa600be79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:21.666024 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238b664-1eab-4926-a76a-ed6aa600be79 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:21.666308 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238b664-1eab-4926-a76a-ed6aa600be79 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:21.666579 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238b664-1eab-4926-a76a-ed6aa600be79 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:21.676193 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8238b664-1eab-4926-a76a-ed6aa600be79 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:21.677647 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8238b664-1eab-4926-a76a-ed6aa600be79 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:21.693225 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-8238b664-1eab-4926-a76a-ed6aa600be79 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:21.693225 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:21.693225 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:21.693225 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:21.693225 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:21.693225 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:21.693225 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:21.693225 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:21.693225 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:21.693225 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:21.693225 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:21.693225 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:21.693225 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 21:50:21.693225 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 21:50:21.693225 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 21:50:21.694147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 21:50:21.694147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:21.694147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:21.694147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:21.694147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:21.694147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:21.694147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:21.694147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:21.694147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:21.694147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:21.694147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:21.694147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:21.694147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:21.694147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:21.694147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:21.694147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:21.694147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:21.696069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 21:50:21.696069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 21:50:21.696069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 21:50:21.696069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 21:50:21.696069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 21:50:21.696069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 21:50:21.696069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:21.696069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:21.696069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 353/699] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:50:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:21.699592 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8238b664-1eab-4926-a76a-ed6aa600be79 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:21.748273 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae14f38-4177-46b1-9c64-adffd472229a None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:21.776595 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:21.800655 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ae3e25-2eb7-4c61-98e8-63ef866a526b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:21.801942 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae3e25-2eb7-4c61-98e8-63ef866a526b None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:21.802890 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae3e25-2eb7-4c61-98e8-63ef866a526b None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:21.803340 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae3e25-2eb7-4c61-98e8-63ef866a526b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:21.818483 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ae3e25-2eb7-4c61-98e8-63ef866a526b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:21.820836 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ae3e25-2eb7-4c61-98e8-63ef866a526b None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:21.823178 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8238b664-1eab-4926-a76a-ed6aa600be79 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:21.843677 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:21.847215 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-23ae3e25-2eb7-4c61-98e8-63ef866a526b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:21.847215 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:21.847215 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:21.847215 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:21.847215 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:21.847215 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:21.847215 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:21.847215 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:21.847215 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:21.847215 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:21.847215 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:21.847215 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:21.847215 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:21.847215 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:21.847215 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:21.848309 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:21.848309 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:21.848309 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:21.848309 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:21.848309 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:21.848309 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:21.848309 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:21.848309 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:21.848309 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:21.848309 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:21.848309 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:21.848309 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:21.848309 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:21.848309 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:21.848309 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:21.848309 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:21.848309 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:21.849332 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:21.849332 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:21.849332 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:21.849332 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:21.851186 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 347/700] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:21.852439 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ae3e25-2eb7-4c61-98e8-63ef866a526b None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:21.869213 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d2d33-ac6d-47df-bb04-ff1cce89a743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:21.870015 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d2d33-ac6d-47df-bb04-ff1cce89a743 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:21.870298 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d2d33-ac6d-47df-bb04-ff1cce89a743 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:21.870573 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d2d33-ac6d-47df-bb04-ff1cce89a743 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:21.881325 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980d2d33-ac6d-47df-bb04-ff1cce89a743 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:21.882869 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980d2d33-ac6d-47df-bb04-ff1cce89a743 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:21.903252 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-980d2d33-ac6d-47df-bb04-ff1cce89a743 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:21.903252 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:21.903252 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:21.903252 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:21.903252 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:21.903252 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:21.903252 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:21.903252 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:21.903252 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:21.903252 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:21.903252 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:21.903252 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:21.903252 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:21.903252 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:21.903252 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:21.904283 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:21.904283 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:21.904283 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:21.904283 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:21.904283 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:21.904283 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:21.904283 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:21.904283 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:21.904283 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:21.904283 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:21.904283 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:21.904283 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:21.904283 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:21.904283 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:21.904283 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:21.904283 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:21.904283 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:21.905200 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:21.905200 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:21.905200 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:21.905200 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:21.905200 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 354/701] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:21.905200 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d2d33-ac6d-47df-bb04-ff1cce89a743 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:21.968827 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ae3e25-2eb7-4c61-98e8-63ef866a526b None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:22.006890 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9226543-767d-4552-9b36-1eeb48aa909d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:22.007505 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9226543-767d-4552-9b36-1eeb48aa909d None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:22.007790 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9226543-767d-4552-9b36-1eeb48aa909d None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:22.008070 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9226543-767d-4552-9b36-1eeb48aa909d None admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:22.008863 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9226543-767d-4552-9b36-1eeb48aa909d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:22.009443 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9226543-767d-4552-9b36-1eeb48aa909d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:22.010684 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9226543-767d-4552-9b36-1eeb48aa909d None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:22.022505 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d2d33-ac6d-47df-bb04-ff1cce89a743 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:22.028624 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 348/702] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jun 7 21:50:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:22.028624 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df87ad-bbfc-4c61-8d37-1fe03936146a None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:22.028624 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df87ad-bbfc-4c61-8d37-1fe03936146a None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:22.028624 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df87ad-bbfc-4c61-8d37-1fe03936146a None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:22.028624 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 349/703] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:22.029236 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33df87ad-bbfc-4c61-8d37-1fe03936146a None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:22.037046 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:22.057353 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aead30b-6059-4f1b-90fe-3f00122b21b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:22.058009 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aead30b-6059-4f1b-90fe-3f00122b21b7 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:22.058290 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aead30b-6059-4f1b-90fe-3f00122b21b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:22.058562 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aead30b-6059-4f1b-90fe-3f00122b21b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:22.059276 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aead30b-6059-4f1b-90fe-3f00122b21b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:22.059606 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aead30b-6059-4f1b-90fe-3f00122b21b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:22.060963 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aead30b-6059-4f1b-90fe-3f00122b21b7 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:22.075206 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 355/704] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:22.080955 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aead30b-6059-4f1b-90fe-3f00122b21b7 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:22.091705 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:22.144458 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33df87ad-bbfc-4c61-8d37-1fe03936146a None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:22.180334 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d8c151-3090-4394-8c78-675a172a16a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:22.180990 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d8c151-3090-4394-8c78-675a172a16a9 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:22.181425 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d8c151-3090-4394-8c78-675a172a16a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:22.181706 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d8c151-3090-4394-8c78-675a172a16a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:22.182996 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d8c151-3090-4394-8c78-675a172a16a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:22.183306 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d8c151-3090-4394-8c78-675a172a16a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:22.184572 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aead30b-6059-4f1b-90fe-3f00122b21b7 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:22.184679 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d8c151-3090-4394-8c78-675a172a16a9 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:22.198399 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 350/705] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:22.203773 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9867c24-4d5f-40d7-822b-1c58966c131a None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:22.204063 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9867c24-4d5f-40d7-822b-1c58966c131a None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:22.204385 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9867c24-4d5f-40d7-822b-1c58966c131a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:22.223234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-a9867c24-4d5f-40d7-822b-1c58966c131a None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:22.224479 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-a9867c24-4d5f-40d7-822b-1c58966c131a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:22.228526 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5651c-0cf4-4a39-8a1c-efb53fefc3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:22.229197 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5651c-0cf4-4a39-8a1c-efb53fefc3d7 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:22.229480 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5651c-0cf4-4a39-8a1c-efb53fefc3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:22.230471 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5651c-0cf4-4a39-8a1c-efb53fefc3d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:22.231351 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e5651c-0cf4-4a39-8a1c-efb53fefc3d7 None admin] RBAC: Adding query filter params (role.id=6b483758dbd44eb4a78df2ce4603d601, scope.project.id=0219dbda54274fdc9bdccbd960c30c6b, user.id=dbe0303ea0764347bfb054870ebba41f) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:22.231678 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e5651c-0cf4-4a39-8a1c-efb53fefc3d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:22.235277 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e5651c-0cf4-4a39-8a1c-efb53fefc3d7 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:22.250449 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 356/706] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:50:22 2022] GET /identity/v3/role_assignments?user.id=dbe0303ea0764347bfb054870ebba41f&scope.project.id=0219dbda54274fdc9bdccbd960c30c6b&role.id=6b483758dbd44eb4a78df2ce4603d601 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:22.250449 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5651c-0cf4-4a39-8a1c-efb53fefc3d7 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:22.303731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:22.317451 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 351/707] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:50:22.318896 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9867c24-4d5f-40d7-822b-1c58966c131a None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:22.387790 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:22.393246 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5651c-0cf4-4a39-8a1c-efb53fefc3d7 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:22.440151 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd1e5ed-3778-4ada-ad52-7d4b5b654f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:22.441081 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1e5ed-3778-4ada-ad52-7d4b5b654f87 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:22.441466 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1e5ed-3778-4ada-ad52-7d4b5b654f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:22.441820 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1e5ed-3778-4ada-ad52-7d4b5b654f87 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:22.450049 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd1e5ed-3778-4ada-ad52-7d4b5b654f87 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:22.451905 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd1e5ed-3778-4ada-ad52-7d4b5b654f87 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:22.461680 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9867c24-4d5f-40d7-822b-1c58966c131a None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:22.470020 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-cbd1e5ed-3778-4ada-ad52-7d4b5b654f87 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:22.470020 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:22.470020 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:22.470020 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:22.470020 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:22.470020 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:22.470020 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:22.470020 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:22.470020 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:22.470020 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:22.470020 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:22.470020 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:22.470020 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 21:50:22.470020 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 21:50:22.470020 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 21:50:22.471162 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 21:50:22.471162 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:22.471162 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:22.471162 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:22.471162 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:22.471162 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:22.471162 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:22.471162 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:22.471162 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:22.471162 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:22.471162 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:22.471162 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:22.471162 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:22.471162 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:22.471162 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:22.471162 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:22.471162 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:22.473018 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 21:50:22.473018 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 21:50:22.473018 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 21:50:22.473018 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 21:50:22.473018 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 21:50:22.473018 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 21:50:22.473018 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:22.473018 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:22.476000 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 357/708] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:50:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:22.476000 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd1e5ed-3778-4ada-ad52-7d4b5b654f87 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:22.502691 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043859c2-fca6-4744-a665-18ea5699e06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:22.503566 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043859c2-fca6-4744-a665-18ea5699e06d None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:22.503844 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043859c2-fca6-4744-a665-18ea5699e06d None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:22.504139 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043859c2-fca6-4744-a665-18ea5699e06d None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:22.504847 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043859c2-fca6-4744-a665-18ea5699e06d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:22.506307 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043859c2-fca6-4744-a665-18ea5699e06d None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:22.535369 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 352/709] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Jun 7 21:50:22 2022] POST /identity/v3/users => generated 254 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 21:50:22.535369 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043859c2-fca6-4744-a665-18ea5699e06d None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:22.690324 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:22.697551 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd1e5ed-3778-4ada-ad52-7d4b5b654f87 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:22.725850 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:22.728677 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043859c2-fca6-4744-a665-18ea5699e06d None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:22.779019 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f3f0d-0886-4037-b5a5-5b6e5fb60522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:22.779899 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f3f0d-0886-4037-b5a5-5b6e5fb60522 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:22.780206 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f3f0d-0886-4037-b5a5-5b6e5fb60522 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:22.780488 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f3f0d-0886-4037-b5a5-5b6e5fb60522 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:22.782171 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61f3f0d-0886-4037-b5a5-5b6e5fb60522 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:22.782723 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61f3f0d-0886-4037-b5a5-5b6e5fb60522 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:22.803047 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-f61f3f0d-0886-4037-b5a5-5b6e5fb60522 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 21:50:22.803047 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:22.803047 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 21:50:22.803047 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 21:50:22.803047 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 21:50:22.803047 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 21:50:22.803047 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 21:50:22.803047 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:50:22.803047 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 21:50:22.803047 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:22.803047 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 21:50:22.803047 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:22.803047 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:22.803047 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:22.803047 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:22.803047 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:22.804218 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:22.804218 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:22.804218 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:22.804218 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:22.804218 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:22.804218 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:22.804218 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:22.804218 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 21:50:22.804218 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 21:50:22.804218 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 21:50:22.804218 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 21:50:22.804218 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:22.804218 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:22.804218 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:22.804218 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:22.804218 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:22.805089 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:22.805089 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:22.805089 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:22.805089 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:22.805089 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:22.805089 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:22.805089 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:22.805089 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:22.805089 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:22.805089 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:22.805089 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:22.805089 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 21:50:22.805089 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:50:22.805089 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 21:50:22.805089 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:22.805089 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 358/710] 10.222.0.13 () {56 vars in 1129 bytes} [Tue Jun 7 21:50:22 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:22.810456 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f3f0d-0886-4037-b5a5-5b6e5fb60522 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:22.811099 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6421c91f-9753-4fc7-8f55-c51ce815424a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:22.812087 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421c91f-9753-4fc7-8f55-c51ce815424a None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:22.812585 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421c91f-9753-4fc7-8f55-c51ce815424a None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:22.813055 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421c91f-9753-4fc7-8f55-c51ce815424a None admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:22.814052 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6421c91f-9753-4fc7-8f55-c51ce815424a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:22.814594 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6421c91f-9753-4fc7-8f55-c51ce815424a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:22.816730 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6421c91f-9753-4fc7-8f55-c51ce815424a None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:22.830354 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 353/711] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jun 7 21:50:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:22.836702 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6421c91f-9753-4fc7-8f55-c51ce815424a None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:22.937136 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f3f0d-0886-4037-b5a5-5b6e5fb60522 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:22.963221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6421c91f-9753-4fc7-8f55-c51ce815424a None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:22.974815 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1938ef3a-f248-43ca-beb0-08653766ca87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:22.975932 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938ef3a-f248-43ca-beb0-08653766ca87 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:22.976412 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938ef3a-f248-43ca-beb0-08653766ca87 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:22.976877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938ef3a-f248-43ca-beb0-08653766ca87 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:22.977796 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1938ef3a-f248-43ca-beb0-08653766ca87 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:22.978295 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1938ef3a-f248-43ca-beb0-08653766ca87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:22.979903 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1938ef3a-f248-43ca-beb0-08653766ca87 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:22.985160 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 359/712] 10.222.0.13 () {56 vars in 1124 bytes} [Tue Jun 7 21:50:22 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:22.994785 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1938ef3a-f248-43ca-beb0-08653766ca87 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:23.011004 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5369bc88-7578-4edb-9bbb-19076394ac49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:23.012580 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5369bc88-7578-4edb-9bbb-19076394ac49 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:23.013243 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5369bc88-7578-4edb-9bbb-19076394ac49 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:23.013754 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5369bc88-7578-4edb-9bbb-19076394ac49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:23.014858 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5369bc88-7578-4edb-9bbb-19076394ac49 None admin] RBAC: Adding query filter params (role.id=2c7f73c184d0406bbf2899465953342b, scope.project.id=0219dbda54274fdc9bdccbd960c30c6b, user.id=7a1261563b6d4e50a4ffeea6a32bdb01) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:23.015424 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5369bc88-7578-4edb-9bbb-19076394ac49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:23.017381 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5369bc88-7578-4edb-9bbb-19076394ac49 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:23.024910 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 354/713] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:50:22 2022] GET /identity/v3/role_assignments?user.id=7a1261563b6d4e50a4ffeea6a32bdb01&scope.project.id=0219dbda54274fdc9bdccbd960c30c6b&role.id=2c7f73c184d0406bbf2899465953342b => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:50:23.034031 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:23.116822 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1938ef3a-f248-43ca-beb0-08653766ca87 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:23.156092 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1576c5-ab5f-4622-a5af-4aa0bb51f233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:23.156770 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1576c5-ab5f-4622-a5af-4aa0bb51f233 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:23.157050 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1576c5-ab5f-4622-a5af-4aa0bb51f233 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:23.157325 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1576c5-ab5f-4622-a5af-4aa0bb51f233 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:23.158043 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1576c5-ab5f-4622-a5af-4aa0bb51f233 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:23.158382 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1576c5-ab5f-4622-a5af-4aa0bb51f233 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:23.160324 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1576c5-ab5f-4622-a5af-4aa0bb51f233 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:23.167143 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 360/714] 10.222.0.13 () {56 vars in 1124 bytes} [Tue Jun 7 21:50:22 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:23.270353 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee36eec9-fdb4-4f60-b061-f69d6f517040 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:23.270546 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee36eec9-fdb4-4f60-b061-f69d6f517040 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:23.270819 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee36eec9-fdb4-4f60-b061-f69d6f517040 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:23.272649 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 355/715] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:23.278781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ab3dd-a908-4b93-9edf-fb2be5486d35 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:23.279078 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ab3dd-a908-4b93-9edf-fb2be5486d35 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:23.279344 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ab3dd-a908-4b93-9edf-fb2be5486d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:23.300269 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-202ab3dd-a908-4b93-9edf-fb2be5486d35 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:23.301736 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-202ab3dd-a908-4b93-9edf-fb2be5486d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:23.390050 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 361/716] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:50:23.413366 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee36eec9-fdb4-4f60-b061-f69d6f517040 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:23.446578 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:23.552693 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee36eec9-fdb4-4f60-b061-f69d6f517040 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:23.605550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea012ce-f9e4-4e10-8738-13a3becfc65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:23.606362 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea012ce-f9e4-4e10-8738-13a3becfc65f None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:23.606645 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea012ce-f9e4-4e10-8738-13a3becfc65f None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:23.606923 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea012ce-f9e4-4e10-8738-13a3becfc65f None admin] PATH_INFO: `/v3/services/network` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:23.607617 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea012ce-f9e4-4e10-8738-13a3becfc65f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:23.608984 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea012ce-f9e4-4e10-8738-13a3becfc65f None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:23.628131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-6ea012ce-f9e4-4e10-8738-13a3becfc65f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 21:50:23.628131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:23.628131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 21:50:23.628131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 21:50:23.628131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 21:50:23.628131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 21:50:23.628131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 21:50:23.628131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:50:23.628131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 21:50:23.628131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:23.628131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 21:50:23.628131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:23.628131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:23.628131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:23.628131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:23.628131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:23.628131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:23.629333 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:23.629333 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:23.629333 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:23.629333 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:23.629333 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:23.629333 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:23.629333 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 21:50:23.629333 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 21:50:23.629333 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 21:50:23.629333 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 21:50:23.629333 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:23.629333 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:23.629333 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:23.629333 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:23.629333 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:23.629333 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:23.630272 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:23.630272 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:23.630272 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:23.630272 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:23.630272 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:23.630272 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:23.630272 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:23.630272 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:23.630272 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:23.630272 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:23.630272 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 21:50:23.630272 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:50:23.630272 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 21:50:23.630272 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:23.634436 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 356/717] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:50:23 2022] GET /identity/v3/services/network => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:23.648599 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ab3dd-a908-4b93-9edf-fb2be5486d35 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:23.679575 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:23.791250 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ab3dd-a908-4b93-9edf-fb2be5486d35 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:23.842766 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f2766-8dcb-4ca0-ac6d-42fba573c15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:23.843363 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f2766-8dcb-4ca0-ac6d-42fba573c15a None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:23.843649 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f2766-8dcb-4ca0-ac6d-42fba573c15a None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:23.844086 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f2766-8dcb-4ca0-ac6d-42fba573c15a None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:23.844801 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6f2766-8dcb-4ca0-ac6d-42fba573c15a None admin] RBAC: Adding query filter params (name=network) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:23.845106 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6f2766-8dcb-4ca0-ac6d-42fba573c15a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:23.847160 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6f2766-8dcb-4ca0-ac6d-42fba573c15a None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:23.852179 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 362/718] 10.222.0.13 () {56 vars in 1112 bytes} [Tue Jun 7 21:50:23 2022] GET /identity/v3/services?name=network => generated 124 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:23.857522 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea012ce-f9e4-4e10-8738-13a3becfc65f None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:24.002671 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:24.015435 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea012ce-f9e4-4e10-8738-13a3becfc65f None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:24.057141 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e37173-07b4-446c-b4de-53372d06cb8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:24.057811 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e37173-07b4-446c-b4de-53372d06cb8a None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:24.058094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e37173-07b4-446c-b4de-53372d06cb8a None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:24.058375 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e37173-07b4-446c-b4de-53372d06cb8a None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:24.059086 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e37173-07b4-446c-b4de-53372d06cb8a None admin] RBAC: Adding query filter params (type=network) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:24.059413 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e37173-07b4-446c-b4de-53372d06cb8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:24.060789 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e37173-07b4-446c-b4de-53372d06cb8a None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:24.071486 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 357/719] 10.222.0.13 () {56 vars in 1112 bytes} [Tue Jun 7 21:50:23 2022] GET /identity/v3/services?type=network => generated 124 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:24.760193 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f7e5b0-8404-4e72-875b-0601cd733235 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:24.760480 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f7e5b0-8404-4e72-875b-0601cd733235 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:24.760762 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f7e5b0-8404-4e72-875b-0601cd733235 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:24.762120 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 363/720] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:24.769555 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190073c2-2f11-466c-8d6d-cca011f7fd9f None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:24.769555 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190073c2-2f11-466c-8d6d-cca011f7fd9f None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:24.769555 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190073c2-2f11-466c-8d6d-cca011f7fd9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:24.798165 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-190073c2-2f11-466c-8d6d-cca011f7fd9f None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:24.799886 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-190073c2-2f11-466c-8d6d-cca011f7fd9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:24.889055 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 358/721] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 21:50:24.923684 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f7e5b0-8404-4e72-875b-0601cd733235 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:25.065068 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:25.093098 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f7e5b0-8404-4e72-875b-0601cd733235 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:25.141286 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b6ea4-f509-4dda-afe5-9efee5e2a34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:25.142085 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b6ea4-f509-4dda-afe5-9efee5e2a34e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:25.142369 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b6ea4-f509-4dda-afe5-9efee5e2a34e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:25.142760 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b6ea4-f509-4dda-afe5-9efee5e2a34e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:25.162140 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066b6ea4-f509-4dda-afe5-9efee5e2a34e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:50:25.169786 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-066b6ea4-f509-4dda-afe5-9efee5e2a34e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:50:25.169786 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:25.169786 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:25.169786 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:25.169786 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:25.169786 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:25.169786 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:25.169786 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:25.169786 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:25.169786 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:25.169786 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:25.169786 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:25.169786 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:50:25.169786 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:50:25.169786 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:50:25.169786 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:50:25.170736 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:50:25.170736 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:50:25.170736 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:50:25.170736 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:50:25.170736 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:25.170736 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:25.170736 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:25.170736 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:25.170736 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:25.170736 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:25.170736 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:25.170736 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:25.170736 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:25.170736 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:25.170736 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:25.170736 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:25.171623 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:25.171623 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:25.171623 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:25.171623 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:25.171623 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:50:25.171623 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:50:25.171623 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:50:25.171623 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:50:25.171623 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:50:25.171623 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:50:25.171623 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:50:25.171623 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:25.173812 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 364/722] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jun 7 21:50:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:25.178937 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190073c2-2f11-466c-8d6d-cca011f7fd9f None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:25.363194 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:25.383415 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190073c2-2f11-466c-8d6d-cca011f7fd9f None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:25.425581 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49984f-171f-4ab4-aacb-2a38bf7803fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:25.426287 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49984f-171f-4ab4-aacb-2a38bf7803fe None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:25.426572 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49984f-171f-4ab4-aacb-2a38bf7803fe None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:25.426848 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49984f-171f-4ab4-aacb-2a38bf7803fe None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:25.427664 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da49984f-171f-4ab4-aacb-2a38bf7803fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:25.427972 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da49984f-171f-4ab4-aacb-2a38bf7803fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:25.429416 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da49984f-171f-4ab4-aacb-2a38bf7803fe None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:25.443820 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 359/723] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:50:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:25.449492 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b6ea4-f509-4dda-afe5-9efee5e2a34e None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:25.576585 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b6ea4-f509-4dda-afe5-9efee5e2a34e None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:25.626283 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7ac9bc-004d-42e4-bd4f-8d3e774120be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:25.627243 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7ac9bc-004d-42e4-bd4f-8d3e774120be None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:25.627572 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7ac9bc-004d-42e4-bd4f-8d3e774120be None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:25.627851 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7ac9bc-004d-42e4-bd4f-8d3e774120be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:25.649388 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7ac9bc-004d-42e4-bd4f-8d3e774120be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:25.650851 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7ac9bc-004d-42e4-bd4f-8d3e774120be None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:25.653620 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:25.669692 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-0e7ac9bc-004d-42e4-bd4f-8d3e774120be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:25.669692 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:25.669692 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:25.669692 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:25.669692 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:25.669692 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:25.669692 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:25.669692 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:25.669692 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:25.669692 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:25.669692 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:25.669692 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:25.669692 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:25.669692 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:25.669692 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:25.670694 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:25.670694 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:25.670694 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:25.670694 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:25.670694 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:25.670694 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:25.670694 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:25.670694 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:25.670694 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:25.670694 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:25.670694 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:25.670694 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:25.670694 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:25.670694 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:25.670694 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:25.670694 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:25.670694 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:25.672070 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:25.672070 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:25.672070 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:25.672070 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:25.673507 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 365/724] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:25.679079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49984f-171f-4ab4-aacb-2a38bf7803fe None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:25.776990 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496553c-011e-4041-9aa7-7ac2b6c366fb None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:25.777361 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496553c-011e-4041-9aa7-7ac2b6c366fb None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:25.777577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496553c-011e-4041-9aa7-7ac2b6c366fb None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:25.779171 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 366/725] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 21:50:25.784957 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e32e7c-f919-4f3f-9753-1313d26c1f40 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:25.785244 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e32e7c-f919-4f3f-9753-1313d26c1f40 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:25.785526 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e32e7c-f919-4f3f-9753-1313d26c1f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:25.803079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-c8e32e7c-f919-4f3f-9753-1313d26c1f40 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:25.804327 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c8e32e7c-f919-4f3f-9753-1313d26c1f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:25.822222 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49984f-171f-4ab4-aacb-2a38bf7803fe None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:25.864307 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c3fcef-b721-4481-996c-a11bef437487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:25.864983 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3fcef-b721-4481-996c-a11bef437487 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:25.865268 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3fcef-b721-4481-996c-a11bef437487 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:25.865548 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3fcef-b721-4481-996c-a11bef437487 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:25.866416 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c3fcef-b721-4481-996c-a11bef437487 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:25.866730 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c3fcef-b721-4481-996c-a11bef437487 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:25.868186 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c3fcef-b721-4481-996c-a11bef437487 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:25.891612 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 367/726] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 21:50:25.894929 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 360/727] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:25.900696 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e32e7c-f919-4f3f-9753-1313d26c1f40 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:25.916490 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c3fcef-b721-4481-996c-a11bef437487 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:25.927656 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:26.055036 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c3fcef-b721-4481-996c-a11bef437487 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:26.100641 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e312b3-773b-4193-982c-68d3855a66f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:26.101265 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e32e7c-f919-4f3f-9753-1313d26c1f40 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:26.101706 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e312b3-773b-4193-982c-68d3855a66f0 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:26.101706 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e312b3-773b-4193-982c-68d3855a66f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:26.101961 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e312b3-773b-4193-982c-68d3855a66f0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:26.117229 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e312b3-773b-4193-982c-68d3855a66f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:50:26.125286 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-c0e312b3-773b-4193-982c-68d3855a66f0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 21:50:26.125286 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:26.125286 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:26.125286 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:26.125286 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:26.125286 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:26.125286 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:26.125286 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:26.125286 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:26.125286 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:26.125286 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:26.125286 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:26.125286 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:50:26.125286 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:50:26.125286 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:50:26.125286 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:50:26.126260 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:50:26.126260 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:50:26.126260 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:50:26.126260 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:50:26.126260 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:26.126260 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:26.126260 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:26.126260 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:26.126260 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:26.126260 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:26.126260 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:26.126260 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:26.126260 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:26.126260 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:26.126260 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:26.126260 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:26.127914 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:26.127914 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:26.127914 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:26.127914 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:26.127914 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:50:26.127914 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:50:26.127914 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:50:26.127914 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:50:26.127914 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:50:26.127914 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:50:26.127914 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 21:50:26.127914 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:26.127914 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 361/728] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jun 7 21:50:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:50:26.128620 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117a6fc-3c46-41fe-bf9e-5aea4a0b1afd None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:26.128620 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117a6fc-3c46-41fe-bf9e-5aea4a0b1afd None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:26.128751 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117a6fc-3c46-41fe-bf9e-5aea4a0b1afd None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:26.134417 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 362/729] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:26.135217 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e117a6fc-3c46-41fe-bf9e-5aea4a0b1afd None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:26.138548 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:26.152936 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9533391d-c28e-498f-8e15-5cf3fae28c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:26.154319 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533391d-c28e-498f-8e15-5cf3fae28c22 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:26.154798 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533391d-c28e-498f-8e15-5cf3fae28c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:26.155419 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533391d-c28e-498f-8e15-5cf3fae28c22 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:26.166164 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9533391d-c28e-498f-8e15-5cf3fae28c22 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:26.168300 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9533391d-c28e-498f-8e15-5cf3fae28c22 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:26.188862 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-9533391d-c28e-498f-8e15-5cf3fae28c22 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 21:50:26.188862 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:26.188862 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:26.188862 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:26.188862 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:26.188862 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:26.188862 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:26.188862 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:26.188862 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:26.188862 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:26.188862 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:26.188862 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:26.188862 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 21:50:26.188862 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 21:50:26.188862 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 21:50:26.188862 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 21:50:26.190115 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:26.190115 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:26.190115 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 21:50:26.190115 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:26.190115 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 21:50:26.190115 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:26.190115 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:26.190115 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:26.190115 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:26.190115 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:26.190115 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:26.190115 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:26.190115 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:26.190115 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:26.190115 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:26.190115 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:26.190115 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:26.190115 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:26.191342 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:26.191342 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:26.191342 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 21:50:26.191342 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 21:50:26.191342 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 21:50:26.191342 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 21:50:26.191342 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 21:50:26.191342 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 21:50:26.191342 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 21:50:26.191342 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:26.194991 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 368/730] 10.222.0.13 () {56 vars in 1093 bytes} [Tue Jun 7 21:50:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:26.196707 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a16b64a-86de-42a9-bbe0-506ca4eb4150 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:26.197066 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a16b64a-86de-42a9-bbe0-506ca4eb4150 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:26.197420 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a16b64a-86de-42a9-bbe0-506ca4eb4150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:26.212163 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:26.217992 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-2a16b64a-86de-42a9-bbe0-506ca4eb4150 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:26.220425 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-2a16b64a-86de-42a9-bbe0-506ca4eb4150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:26.262684 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e117a6fc-3c46-41fe-bf9e-5aea4a0b1afd None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:26.293724 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 369/731] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:50:26.294858 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a16b64a-86de-42a9-bbe0-506ca4eb4150 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:26.303271 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39de250-1a62-4854-b3d3-3e4691ae4c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:26.304832 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39de250-1a62-4854-b3d3-3e4691ae4c76 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:26.305330 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39de250-1a62-4854-b3d3-3e4691ae4c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:26.306272 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39de250-1a62-4854-b3d3-3e4691ae4c76 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:26.307684 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39de250-1a62-4854-b3d3-3e4691ae4c76 None admin] RBAC: Adding query filter params (name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:26.308490 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39de250-1a62-4854-b3d3-3e4691ae4c76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:26.310343 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39de250-1a62-4854-b3d3-3e4691ae4c76 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:26.321711 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 363/732] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Jun 7 21:50:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:26.323388 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6a0ae-807f-4938-9e82-59ed040f7a06 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:26.323896 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6a0ae-807f-4938-9e82-59ed040f7a06 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:26.324383 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6a0ae-807f-4938-9e82-59ed040f7a06 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:26.326055 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 364/733] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:26.326934 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb6a0ae-807f-4938-9e82-59ed040f7a06 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:26.406947 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a16b64a-86de-42a9-bbe0-506ca4eb4150 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:26.424334 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:26.437184 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb6a0ae-807f-4938-9e82-59ed040f7a06 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:26.447070 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cd98bd-c07f-4246-b92f-5511a7e347df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:26.448421 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd98bd-c07f-4246-b92f-5511a7e347df None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:26.448896 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd98bd-c07f-4246-b92f-5511a7e347df None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:26.449354 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd98bd-c07f-4246-b92f-5511a7e347df None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:26.450346 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cd98bd-c07f-4246-b92f-5511a7e347df None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:26.450919 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cd98bd-c07f-4246-b92f-5511a7e347df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:26.452672 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cd98bd-c07f-4246-b92f-5511a7e347df None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:26.465558 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:26.475595 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c88810-6d52-4351-8af0-a2d69670b078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:26.476266 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c88810-6d52-4351-8af0-a2d69670b078 None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:26.476551 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c88810-6d52-4351-8af0-a2d69670b078 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:26.476827 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c88810-6d52-4351-8af0-a2d69670b078 None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:26.477609 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c88810-6d52-4351-8af0-a2d69670b078 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:26.479000 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c88810-6d52-4351-8af0-a2d69670b078 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:26.479829 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 370/734] 10.222.0.13 () {56 vars in 1133 bytes} [Tue Jun 7 21:50:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:26.481352 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b24bfa-88e3-4f47-b33e-0e0657edd775 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:26.481645 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b24bfa-88e3-4f47-b33e-0e0657edd775 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:26.481925 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b24bfa-88e3-4f47-b33e-0e0657edd775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:26.496551 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 365/735] 10.222.0.13 () {60 vars in 1141 bytes} [Tue Jun 7 21:50:26 2022] POST /identity/v3/services => generated 251 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 21:50:26.497464 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c88810-6d52-4351-8af0-a2d69670b078 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:26.520733 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-b0b24bfa-88e3-4f47-b33e-0e0657edd775 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:26.522119 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-b0b24bfa-88e3-4f47-b33e-0e0657edd775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:26.610311 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 371/736] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:26 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:50:26.611289 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b24bfa-88e3-4f47-b33e-0e0657edd775 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:26.663572 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c88810-6d52-4351-8af0-a2d69670b078 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:26.718931 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f27186-0b06-4558-b7a0-7351c17ad8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:26.720048 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f27186-0b06-4558-b7a0-7351c17ad8bf None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:26.720336 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f27186-0b06-4558-b7a0-7351c17ad8bf None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:26.720657 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f27186-0b06-4558-b7a0-7351c17ad8bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:26.738346 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f27186-0b06-4558-b7a0-7351c17ad8bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:26.740152 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f27186-0b06-4558-b7a0-7351c17ad8bf None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:26.741079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:26.744894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b24bfa-88e3-4f47-b33e-0e0657edd775 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:26.766925 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-e1f27186-0b06-4558-b7a0-7351c17ad8bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:26.766925 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:26.766925 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:26.766925 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:26.766925 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:26.766925 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:26.766925 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:26.766925 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:26.766925 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:26.766925 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:26.766925 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:26.766925 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:26.766925 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:26.766925 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:26.766925 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:26.768815 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:26.768815 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:26.768815 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:26.768815 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:26.768815 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:26.768815 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:26.768815 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:26.768815 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:26.768815 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:26.768815 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:26.768815 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:26.768815 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:26.768815 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:26.768815 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:26.768815 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:26.768815 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:26.768815 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:26.769730 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:26.769730 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:26.769730 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:26.769730 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:26.770381 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 366/737] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:26.771278 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f27186-0b06-4558-b7a0-7351c17ad8bf None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:26.816738 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d2fc1f-debd-48e6-822e-499e1fb46c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:26.818461 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2fc1f-debd-48e6-822e-499e1fb46c80 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:26.818751 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2fc1f-debd-48e6-822e-499e1fb46c80 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:26.819032 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2fc1f-debd-48e6-822e-499e1fb46c80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:26.823927 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:26.834706 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d2fc1f-debd-48e6-822e-499e1fb46c80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:26.837091 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d2fc1f-debd-48e6-822e-499e1fb46c80 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:26.863414 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-38d2fc1f-debd-48e6-822e-499e1fb46c80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:26.863414 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:26.863414 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:26.863414 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:26.863414 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:26.863414 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:26.863414 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:26.863414 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:26.863414 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:26.863414 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:26.863414 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:26.863414 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:26.863414 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:26.863414 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:26.863414 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:26.864363 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:26.864363 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:26.864363 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:26.864363 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:26.864363 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:26.864363 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:26.864363 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:26.864363 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:26.864363 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:26.864363 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:26.864363 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:26.864363 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:26.864363 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:26.864363 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:26.864363 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:26.864363 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:26.864363 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:26.865266 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:26.865266 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:26.865266 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:26.865266 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:26.865266 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 372/738] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:26.865562 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d2fc1f-debd-48e6-822e-499e1fb46c80 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:26.900121 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f27186-0b06-4558-b7a0-7351c17ad8bf None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:26.938644 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be91da72-e73e-4ef2-b742-47d3dbffbb26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:26.940161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be91da72-e73e-4ef2-b742-47d3dbffbb26 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:26.940723 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be91da72-e73e-4ef2-b742-47d3dbffbb26 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:26.941216 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be91da72-e73e-4ef2-b742-47d3dbffbb26 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:26.954263 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be91da72-e73e-4ef2-b742-47d3dbffbb26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:50:26.963557 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-be91da72-e73e-4ef2-b742-47d3dbffbb26 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:50:26.963557 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:26.963557 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:26.963557 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:26.963557 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:26.963557 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:26.963557 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:26.963557 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:26.963557 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:26.963557 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:26.963557 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:26.963557 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:26.963557 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:50:26.963557 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:50:26.963557 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:50:26.963557 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:50:26.964517 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:50:26.964517 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:50:26.964517 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:50:26.964517 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:50:26.964517 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:26.964517 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:26.964517 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:26.964517 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:26.964517 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:26.964517 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:26.964517 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:26.964517 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:26.964517 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:26.964517 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:26.964517 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:26.964517 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:26.965353 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:26.965353 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:26.965353 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:26.965353 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:26.965353 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:50:26.965353 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:50:26.965353 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:50:26.965353 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:50:26.965353 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:50:26.965353 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:50:26.965353 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:50:26.965353 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:26.968231 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 367/739] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jun 7 21:50:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:26.969259 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be91da72-e73e-4ef2-b742-47d3dbffbb26 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:27.005356 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d2fc1f-debd-48e6-822e-499e1fb46c80 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:27.042434 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d5757e-c1d9-410d-bf2a-9fa6c3ee5513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:27.043504 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d5757e-c1d9-410d-bf2a-9fa6c3ee5513 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:27.044183 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d5757e-c1d9-410d-bf2a-9fa6c3ee5513 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:27.044675 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d5757e-c1d9-410d-bf2a-9fa6c3ee5513 None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:27.045593 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d5757e-c1d9-410d-bf2a-9fa6c3ee5513 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:27.046093 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d5757e-c1d9-410d-bf2a-9fa6c3ee5513 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:27.047696 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d5757e-c1d9-410d-bf2a-9fa6c3ee5513 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:27.065566 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 373/740] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:27.066783 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d5757e-c1d9-410d-bf2a-9fa6c3ee5513 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:27.082612 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:27.100436 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:27.106144 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be91da72-e73e-4ef2-b742-47d3dbffbb26 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:27.144597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f15a27-d9a3-4ddf-b0c6-e717cb958a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:27.145556 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f15a27-d9a3-4ddf-b0c6-e717cb958a3b None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:27.145837 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f15a27-d9a3-4ddf-b0c6-e717cb958a3b None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:27.146113 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f15a27-d9a3-4ddf-b0c6-e717cb958a3b None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:27.146840 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f15a27-d9a3-4ddf-b0c6-e717cb958a3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:27.147143 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f15a27-d9a3-4ddf-b0c6-e717cb958a3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:27.148530 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f15a27-d9a3-4ddf-b0c6-e717cb958a3b None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:27.161429 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 368/741] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:27.162256 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f15a27-d9a3-4ddf-b0c6-e717cb958a3b None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:27.190573 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d5757e-c1d9-410d-bf2a-9fa6c3ee5513 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:27.227184 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8055b9ee-d6b1-4a34-84ee-d6dd8f3737ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:27.227978 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055b9ee-d6b1-4a34-84ee-d6dd8f3737ac None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:27.228270 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055b9ee-d6b1-4a34-84ee-d6dd8f3737ac None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:27.228545 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055b9ee-d6b1-4a34-84ee-d6dd8f3737ac None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:27.229291 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8055b9ee-d6b1-4a34-84ee-d6dd8f3737ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:27.229597 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8055b9ee-d6b1-4a34-84ee-d6dd8f3737ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:27.230975 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8055b9ee-d6b1-4a34-84ee-d6dd8f3737ac None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:27.244541 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 374/742] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:50:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:27.245647 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf07bb6-bebe-470a-b4bf-4405c1f9e41c None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:27.245941 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf07bb6-bebe-470a-b4bf-4405c1f9e41c None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:27.246218 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf07bb6-bebe-470a-b4bf-4405c1f9e41c None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:27.247773 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 375/743] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:27.248477 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf07bb6-bebe-470a-b4bf-4405c1f9e41c None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:27.272397 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f15a27-d9a3-4ddf-b0c6-e717cb958a3b None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:27.308629 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a23d47-89a0-4afa-b764-4a8bf35a5356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:27.309438 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a23d47-89a0-4afa-b764-4a8bf35a5356 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:27.309720 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a23d47-89a0-4afa-b764-4a8bf35a5356 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:27.309993 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a23d47-89a0-4afa-b764-4a8bf35a5356 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:27.317956 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a23d47-89a0-4afa-b764-4a8bf35a5356 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:27.319430 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a23d47-89a0-4afa-b764-4a8bf35a5356 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:27.329182 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:27.334870 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-34a23d47-89a0-4afa-b764-4a8bf35a5356 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 21:50:27.334870 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:27.334870 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:27.334870 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:27.334870 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:27.334870 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:27.334870 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:27.334870 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:27.334870 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:27.334870 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:27.334870 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:27.334870 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:27.334870 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 21:50:27.334870 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 21:50:27.334870 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 21:50:27.334870 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 21:50:27.336082 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:27.336082 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:27.336082 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 21:50:27.336082 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:27.336082 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 21:50:27.336082 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:27.336082 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:27.336082 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:27.336082 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:27.336082 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:27.336082 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:27.336082 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:27.336082 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:27.336082 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:27.336082 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:27.336082 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:27.336082 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:27.336082 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:27.337049 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:27.337049 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:27.337049 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 21:50:27.337049 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 21:50:27.337049 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 21:50:27.337049 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 21:50:27.337049 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 21:50:27.337049 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 21:50:27.337049 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 21:50:27.337049 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:27.337049 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 369/744] 10.222.0.13 () {56 vars in 1117 bytes} [Tue Jun 7 21:50:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:27.337049 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a23d47-89a0-4afa-b764-4a8bf35a5356 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:27.358566 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:27.367733 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf07bb6-bebe-470a-b4bf-4405c1f9e41c None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:27.420310 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f861d1-016e-443e-a377-043665b1d179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:27.421129 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f861d1-016e-443e-a377-043665b1d179 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:27.421456 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f861d1-016e-443e-a377-043665b1d179 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:27.421709 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f861d1-016e-443e-a377-043665b1d179 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:27.429792 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f861d1-016e-443e-a377-043665b1d179 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:27.431264 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f861d1-016e-443e-a377-043665b1d179 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:27.445978 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-33f861d1-016e-443e-a377-043665b1d179 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:27.445978 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:27.445978 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:27.445978 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:27.445978 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:27.445978 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:27.445978 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:27.445978 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:27.445978 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:27.445978 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:27.445978 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:27.445978 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:27.445978 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 21:50:27.445978 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 21:50:27.445978 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 21:50:27.446883 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 21:50:27.446883 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:27.446883 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:27.446883 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:27.446883 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:27.446883 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:27.446883 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:27.446883 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:27.446883 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:27.446883 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:27.446883 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:27.446883 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:27.446883 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:27.446883 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:27.446883 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:27.446883 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:27.446883 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:27.448338 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 21:50:27.448338 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 21:50:27.448338 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 21:50:27.448338 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 21:50:27.448338 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 21:50:27.448338 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 21:50:27.448338 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:27.448338 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:27.448338 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 376/745] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:50:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:27.448827 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b63a6-d7f0-48c2-a60c-38a125921d39 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:27.448827 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b63a6-d7f0-48c2-a60c-38a125921d39 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:27.448957 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b63a6-d7f0-48c2-a60c-38a125921d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:27.461860 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a23d47-89a0-4afa-b764-4a8bf35a5356 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:27.466640 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-624b63a6-d7f0-48c2-a60c-38a125921d39 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:27.467871 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-624b63a6-d7f0-48c2-a60c-38a125921d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:27.500809 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ec8dd2-d61f-439d-8001-13f1cdbc0283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:27.501600 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec8dd2-d61f-439d-8001-13f1cdbc0283 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:27.502409 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec8dd2-d61f-439d-8001-13f1cdbc0283 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:27.502707 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec8dd2-d61f-439d-8001-13f1cdbc0283 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:27.513362 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ec8dd2-d61f-439d-8001-13f1cdbc0283 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:27.514808 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ec8dd2-d61f-439d-8001-13f1cdbc0283 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:27.534428 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-d5ec8dd2-d61f-439d-8001-13f1cdbc0283 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:27.534428 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:27.534428 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:27.534428 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:27.534428 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:27.534428 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:27.534428 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:27.534428 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:27.534428 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:27.534428 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:27.534428 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:27.534428 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:27.534428 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:27.534428 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:27.534428 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:27.535408 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:27.535408 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:27.535408 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:27.535408 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:27.535408 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:27.535408 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:27.535408 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:27.535408 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:27.535408 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:27.535408 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:27.535408 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:27.535408 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:27.535408 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:27.535408 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:27.535408 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:27.535408 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:27.535408 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:27.536306 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:27.536306 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:27.536306 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:27.536306 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:27.536306 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 370/746] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:27.536582 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ec8dd2-d61f-439d-8001-13f1cdbc0283 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:27.542773 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 377/747] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:27 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 21:50:27.544072 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b63a6-d7f0-48c2-a60c-38a125921d39 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:27.612174 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:27.617125 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:27.653502 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ec8dd2-d61f-439d-8001-13f1cdbc0283 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:27.655904 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b63a6-d7f0-48c2-a60c-38a125921d39 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:27.690496 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c5cbbe-8dd9-4582-8eb1-47abf96ae524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:27.691495 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c5cbbe-8dd9-4582-8eb1-47abf96ae524 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:27.691866 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c5cbbe-8dd9-4582-8eb1-47abf96ae524 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:27.692252 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c5cbbe-8dd9-4582-8eb1-47abf96ae524 None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:27.693116 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c5cbbe-8dd9-4582-8eb1-47abf96ae524 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:27.693496 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c5cbbe-8dd9-4582-8eb1-47abf96ae524 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:27.693672 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8137491-a3a8-4ed5-a255-cf36be40a8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:27.694324 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8137491-a3a8-4ed5-a255-cf36be40a8a1 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:27.694604 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8137491-a3a8-4ed5-a255-cf36be40a8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:27.694876 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8137491-a3a8-4ed5-a255-cf36be40a8a1 None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:27.695270 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c5cbbe-8dd9-4582-8eb1-47abf96ae524 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:27.695683 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8137491-a3a8-4ed5-a255-cf36be40a8a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:27.696038 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8137491-a3a8-4ed5-a255-cf36be40a8a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:27.697424 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8137491-a3a8-4ed5-a255-cf36be40a8a1 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:27.710159 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 378/748] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jun 7 21:50:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:27.710771 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8137491-a3a8-4ed5-a255-cf36be40a8a1 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:27.725350 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 371/749] 10.222.0.13 () {56 vars in 1149 bytes} [Tue Jun 7 21:50:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:27.725350 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c5cbbe-8dd9-4582-8eb1-47abf96ae524 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:27.829594 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8137491-a3a8-4ed5-a255-cf36be40a8a1 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:27.855122 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:27.855437 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c5cbbe-8dd9-4582-8eb1-47abf96ae524 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:27.866583 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c100097-a462-4df9-b337-044bc56e9dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:27.867251 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c100097-a462-4df9-b337-044bc56e9dea None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:27.867570 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c100097-a462-4df9-b337-044bc56e9dea None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:27.867847 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c100097-a462-4df9-b337-044bc56e9dea None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:27.868646 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c100097-a462-4df9-b337-044bc56e9dea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:27.868871 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c100097-a462-4df9-b337-044bc56e9dea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:27.870246 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c100097-a462-4df9-b337-044bc56e9dea None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:27.883654 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 379/750] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:27.884472 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c100097-a462-4df9-b337-044bc56e9dea None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:27.889287 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:27.901168 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a7022-9933-4e7c-b6dd-a87b3dda44f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:27.902460 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a7022-9933-4e7c-b6dd-a87b3dda44f1 None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:27.903198 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a7022-9933-4e7c-b6dd-a87b3dda44f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:27.904134 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a7022-9933-4e7c-b6dd-a87b3dda44f1 None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:27.905108 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a7022-9933-4e7c-b6dd-a87b3dda44f1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:27.907266 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a7022-9933-4e7c-b6dd-a87b3dda44f1 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:27.924742 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 372/751] 10.222.0.13 () {60 vars in 1141 bytes} [Tue Jun 7 21:50:27 2022] POST /identity/v3/services => generated 240 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 21:50:27.925733 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a7022-9933-4e7c-b6dd-a87b3dda44f1 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:28.009112 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c100097-a462-4df9-b337-044bc56e9dea None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:28.068934 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523eb9cc-ef99-4a73-973d-04bbb22c7d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:28.070146 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523eb9cc-ef99-4a73-973d-04bbb22c7d9c None admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:28.070856 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523eb9cc-ef99-4a73-973d-04bbb22c7d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:28.071285 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523eb9cc-ef99-4a73-973d-04bbb22c7d9c None admin] PATH_INFO: `/v3/projects/0219dbda54274fdc9bdccbd960c30c6b/users/dbe0303ea0764347bfb054870ebba41f/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:28.074473 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523eb9cc-ef99-4a73-973d-04bbb22c7d9c None admin] RBAC: Authorizing `identity:create_grant(project_id=0219dbda54274fdc9bdccbd960c30c6b, user_id=dbe0303ea0764347bfb054870ebba41f, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:28.076940 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523eb9cc-ef99-4a73-973d-04bbb22c7d9c None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:28.097412 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 380/752] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jun 7 21:50:27 2022] PUT /identity/v3/projects/0219dbda54274fdc9bdccbd960c30c6b/users/dbe0303ea0764347bfb054870ebba41f/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 21:50:28.098649 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523eb9cc-ef99-4a73-973d-04bbb22c7d9c None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:28.104104 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a7022-9933-4e7c-b6dd-a87b3dda44f1 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:28.155497 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:28.173306 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e4575-cb05-438b-961d-3bc5eee40d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:28.174944 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e4575-cb05-438b-961d-3bc5eee40d74 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:28.175429 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e4575-cb05-438b-961d-3bc5eee40d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:28.176454 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e4575-cb05-438b-961d-3bc5eee40d74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:28.249371 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523eb9cc-ef99-4a73-973d-04bbb22c7d9c None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:28.262073 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:28.290470 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1e4575-cb05-438b-961d-3bc5eee40d74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:28.295299 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1e4575-cb05-438b-961d-3bc5eee40d74 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:28.329493 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92d0b6e-7aad-42a8-8d37-a0c042c9ef91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:28.330462 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92d0b6e-7aad-42a8-8d37-a0c042c9ef91 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:28.330778 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92d0b6e-7aad-42a8-8d37-a0c042c9ef91 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:28.331104 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92d0b6e-7aad-42a8-8d37-a0c042c9ef91 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:28.345127 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92d0b6e-7aad-42a8-8d37-a0c042c9ef91 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:28.347676 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92d0b6e-7aad-42a8-8d37-a0c042c9ef91 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:28.364645 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-d92d0b6e-7aad-42a8-8d37-a0c042c9ef91 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 21:50:28.364645 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:28.364645 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:28.364645 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:28.364645 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:28.364645 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:28.364645 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:28.364645 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:28.364645 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:28.364645 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:28.364645 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:28.364645 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:28.364645 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 21:50:28.364645 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 21:50:28.364645 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 21:50:28.364645 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 21:50:28.365690 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:28.365690 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:28.365690 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 21:50:28.365690 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:28.365690 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 21:50:28.365690 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:28.365690 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:28.365690 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:28.365690 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:28.365690 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:28.365690 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:28.365690 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:28.365690 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:28.365690 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:28.365690 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:28.365690 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:28.365690 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:28.365690 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:28.366629 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:28.366629 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:28.366629 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 21:50:28.366629 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 21:50:28.366629 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 21:50:28.366629 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 21:50:28.366629 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 21:50:28.366629 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 21:50:28.366629 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 21:50:28.366629 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:28.366629 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 381/753] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jun 7 21:50:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:28.372107 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92d0b6e-7aad-42a8-8d37-a0c042c9ef91 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:28.447426 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 373/754] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:50:27 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 21:50:28.450077 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e4575-cb05-438b-961d-3bc5eee40d74 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:28.499540 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:28.504680 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92d0b6e-7aad-42a8-8d37-a0c042c9ef91 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:28.559926 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:28.574019 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e4575-cb05-438b-961d-3bc5eee40d74 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:28.600802 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfed401c-b1eb-4503-8db3-34928bef5137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:28.601881 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfed401c-b1eb-4503-8db3-34928bef5137 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:28.602364 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfed401c-b1eb-4503-8db3-34928bef5137 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:28.602859 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfed401c-b1eb-4503-8db3-34928bef5137 None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:28.604036 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfed401c-b1eb-4503-8db3-34928bef5137 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:28.604567 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfed401c-b1eb-4503-8db3-34928bef5137 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:28.606552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfed401c-b1eb-4503-8db3-34928bef5137 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:28.624281 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0482ac92-654b-44b9-9f0f-ea6da7883ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:28.625091 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0482ac92-654b-44b9-9f0f-ea6da7883ef9 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:28.625375 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0482ac92-654b-44b9-9f0f-ea6da7883ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:28.625651 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0482ac92-654b-44b9-9f0f-ea6da7883ef9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:28.640858 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 382/755] 10.222.0.13 () {56 vars in 1135 bytes} [Tue Jun 7 21:50:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:50:28.647538 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfed401c-b1eb-4503-8db3-34928bef5137 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:28.656661 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0482ac92-654b-44b9-9f0f-ea6da7883ef9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:28.658868 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0482ac92-654b-44b9-9f0f-ea6da7883ef9 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:28.701634 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-0482ac92-654b-44b9-9f0f-ea6da7883ef9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:28.701634 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:28.701634 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:28.701634 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:28.701634 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:28.701634 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:28.701634 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:28.701634 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:28.701634 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:28.701634 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:28.701634 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:28.701634 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:28.701634 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:28.701634 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:28.701634 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:28.702606 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:28.702606 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:28.702606 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:28.702606 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:28.702606 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:28.702606 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:28.702606 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:28.702606 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:28.702606 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:28.702606 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:28.702606 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:28.702606 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:28.702606 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:28.702606 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:28.702606 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:28.702606 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:28.702606 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:28.703603 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:28.703603 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:28.703603 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:28.703603 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:28.703603 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 374/756] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:28.723754 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0482ac92-654b-44b9-9f0f-ea6da7883ef9 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:28.763464 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfed401c-b1eb-4503-8db3-34928bef5137 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:28.802669 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e806d7a0-db54-40a9-95f0-c56053ed87b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:28.803347 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e806d7a0-db54-40a9-95f0-c56053ed87b0 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:28.803636 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e806d7a0-db54-40a9-95f0-c56053ed87b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:28.803916 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e806d7a0-db54-40a9-95f0-c56053ed87b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:28.835827 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:28.880702 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e806d7a0-db54-40a9-95f0-c56053ed87b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:28.882343 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e806d7a0-db54-40a9-95f0-c56053ed87b0 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:28.989583 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0482ac92-654b-44b9-9f0f-ea6da7883ef9 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:29.008272 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 383/757] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:50:28 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 21:50:29.009803 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ab371-9b42-4896-8496-d01f9baac3ab None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:29.010097 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ab371-9b42-4896-8496-d01f9baac3ab None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:29.010379 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ab371-9b42-4896-8496-d01f9baac3ab None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:29.012831 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 384/758] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:29.013691 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ab371-9b42-4896-8496-d01f9baac3ab None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:29.030678 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66fad54-f045-46f8-9181-d957a19e5411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:29.031643 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66fad54-f045-46f8-9181-d957a19e5411 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:29.031926 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66fad54-f045-46f8-9181-d957a19e5411 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:29.032202 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66fad54-f045-46f8-9181-d957a19e5411 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:29.033135 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66fad54-f045-46f8-9181-d957a19e5411 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:29.033576 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66fad54-f045-46f8-9181-d957a19e5411 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:29.035829 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66fad54-f045-46f8-9181-d957a19e5411 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:29.046620 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:29.052330 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 375/759] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:29.053921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed08924b-ab89-406e-aeee-a1a22245fb04 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:29.054515 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed08924b-ab89-406e-aeee-a1a22245fb04 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:29.054937 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed08924b-ab89-406e-aeee-a1a22245fb04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:29.075408 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-ed08924b-ab89-406e-aeee-a1a22245fb04 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:29.076881 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-ed08924b-ab89-406e-aeee-a1a22245fb04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:29.090211 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:29.149199 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ab371-9b42-4896-8496-d01f9baac3ab None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:29.160654 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 376/760] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:50:29.161553 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed08924b-ab89-406e-aeee-a1a22245fb04 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:29.211043 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f205ce27-8138-4b75-817d-771d041f2243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:29.211996 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205ce27-8138-4b75-817d-771d041f2243 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:29.212282 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205ce27-8138-4b75-817d-771d041f2243 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:29.212555 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205ce27-8138-4b75-817d-771d041f2243 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:29.224953 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f205ce27-8138-4b75-817d-771d041f2243 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:29.226485 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f205ce27-8138-4b75-817d-771d041f2243 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:29.245987 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-f205ce27-8138-4b75-817d-771d041f2243 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:29.245987 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:29.245987 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:29.245987 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:29.245987 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:29.245987 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:29.245987 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:29.245987 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:29.245987 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:29.245987 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:29.245987 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:29.245987 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:29.245987 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:29.245987 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:29.245987 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:29.246933 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:29.246933 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:29.246933 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:29.246933 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:29.246933 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:29.246933 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:29.246933 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:29.246933 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:29.246933 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:29.246933 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:29.246933 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:29.246933 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:29.246933 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:29.246933 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:29.246933 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:29.246933 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:29.246933 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:29.247900 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:29.247900 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:29.247900 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:29.247900 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:29.247900 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 385/761] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:29.248633 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f205ce27-8138-4b75-817d-771d041f2243 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:29.270851 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed08924b-ab89-406e-aeee-a1a22245fb04 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:29.304840 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:29.311693 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f57643-77db-4cd1-9d85-ac139c8678e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:29.312611 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f57643-77db-4cd1-9d85-ac139c8678e0 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:29.312920 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f57643-77db-4cd1-9d85-ac139c8678e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:29.313221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f57643-77db-4cd1-9d85-ac139c8678e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:29.320954 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f57643-77db-4cd1-9d85-ac139c8678e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:29.322419 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f57643-77db-4cd1-9d85-ac139c8678e0 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:29.338511 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-96f57643-77db-4cd1-9d85-ac139c8678e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:29.338511 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:29.338511 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:29.338511 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:29.338511 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:29.338511 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:29.338511 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:29.338511 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:29.338511 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:29.338511 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:29.338511 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:29.338511 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:29.338511 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 21:50:29.338511 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 21:50:29.338511 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 21:50:29.340702 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 21:50:29.340702 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:29.340702 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:29.340702 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:29.340702 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:29.340702 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:29.340702 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:29.340702 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:29.340702 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:29.340702 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:29.340702 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:29.340702 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:29.340702 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:29.340702 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:29.340702 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:29.340702 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:29.340702 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:29.342990 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 21:50:29.342990 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 21:50:29.342990 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 21:50:29.342990 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 21:50:29.342990 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 21:50:29.342990 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 21:50:29.342990 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:29.342990 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:29.342990 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 377/762] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:50:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:29.342990 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f57643-77db-4cd1-9d85-ac139c8678e0 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:29.364432 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f205ce27-8138-4b75-817d-771d041f2243 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:29.376409 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:29.403197 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a91a9a7-cc49-4e7b-803c-8c3e6dac2b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:29.404038 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91a9a7-cc49-4e7b-803c-8c3e6dac2b41 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:29.404317 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91a9a7-cc49-4e7b-803c-8c3e6dac2b41 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:29.404694 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91a9a7-cc49-4e7b-803c-8c3e6dac2b41 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:29.405452 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a91a9a7-cc49-4e7b-803c-8c3e6dac2b41 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:29.406817 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a91a9a7-cc49-4e7b-803c-8c3e6dac2b41 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:29.420715 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-1a91a9a7-cc49-4e7b-803c-8c3e6dac2b41 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 21:50:29.420715 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:29.420715 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 21:50:29.420715 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 21:50:29.420715 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 21:50:29.420715 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 21:50:29.420715 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 21:50:29.420715 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:50:29.420715 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 21:50:29.420715 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:29.420715 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 21:50:29.420715 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:29.420715 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:29.420715 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:29.420715 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:29.420715 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:29.422904 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:29.422904 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:29.422904 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:29.422904 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:29.422904 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:29.422904 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:29.422904 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:29.422904 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 21:50:29.422904 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 21:50:29.422904 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 21:50:29.422904 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 21:50:29.422904 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:29.422904 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:29.422904 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:29.422904 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:29.422904 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:29.427380 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:29.427380 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:29.427380 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:29.427380 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:29.427380 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:29.427380 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:29.427380 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:29.427380 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:29.427380 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:29.427380 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:29.427380 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:29.427380 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 21:50:29.427380 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:50:29.427380 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 21:50:29.427380 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:29.427380 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 386/763] 10.222.0.13 () {56 vars in 1129 bytes} [Tue Jun 7 21:50:29 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:29.429498 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a91a9a7-cc49-4e7b-803c-8c3e6dac2b41 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:29.461674 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f57643-77db-4cd1-9d85-ac139c8678e0 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:29.500889 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa1ea7-98f8-4cee-af29-1047c5de1d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:29.501712 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa1ea7-98f8-4cee-af29-1047c5de1d37 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:29.502030 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa1ea7-98f8-4cee-af29-1047c5de1d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:29.502344 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa1ea7-98f8-4cee-af29-1047c5de1d37 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:29.503082 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fa1ea7-98f8-4cee-af29-1047c5de1d37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:29.503422 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fa1ea7-98f8-4cee-af29-1047c5de1d37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:29.504939 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fa1ea7-98f8-4cee-af29-1047c5de1d37 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:29.518855 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 378/764] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:29.519942 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa1ea7-98f8-4cee-af29-1047c5de1d37 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:29.537842 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a91a9a7-cc49-4e7b-803c-8c3e6dac2b41 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:29.574961 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:29.576292 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d8cfe-1142-4409-a320-0288ad6f0cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:29.576957 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d8cfe-1142-4409-a320-0288ad6f0cf1 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:29.577236 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d8cfe-1142-4409-a320-0288ad6f0cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:29.577616 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d8cfe-1142-4409-a320-0288ad6f0cf1 None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:29.578500 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087d8cfe-1142-4409-a320-0288ad6f0cf1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:29.578911 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087d8cfe-1142-4409-a320-0288ad6f0cf1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:29.581528 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087d8cfe-1142-4409-a320-0288ad6f0cf1 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:29.595717 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 387/765] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jun 7 21:50:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:29.597182 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d8cfe-1142-4409-a320-0288ad6f0cf1 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:29.634099 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa1ea7-98f8-4cee-af29-1047c5de1d37 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:29.635144 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:29.675609 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a9ad15-5413-4e55-be35-3bd84ac2abcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:29.676713 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9ad15-5413-4e55-be35-3bd84ac2abcc None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:29.677251 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9ad15-5413-4e55-be35-3bd84ac2abcc None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:29.677676 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9ad15-5413-4e55-be35-3bd84ac2abcc None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:29.678634 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a9ad15-5413-4e55-be35-3bd84ac2abcc None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:29.679178 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a9ad15-5413-4e55-be35-3bd84ac2abcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:29.680735 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a9ad15-5413-4e55-be35-3bd84ac2abcc None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:29.686319 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 379/766] 10.222.0.13 () {56 vars in 1124 bytes} [Tue Jun 7 21:50:29 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:29.687299 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a9ad15-5413-4e55-be35-3bd84ac2abcc None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:29.714196 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d8cfe-1142-4409-a320-0288ad6f0cf1 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:29.753909 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc08f67-5dd4-44d3-a3b9-87f5ef7a268a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:29.754711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc08f67-5dd4-44d3-a3b9-87f5ef7a268a None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:29.754991 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc08f67-5dd4-44d3-a3b9-87f5ef7a268a None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:29.755264 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc08f67-5dd4-44d3-a3b9-87f5ef7a268a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:29.767348 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc08f67-5dd4-44d3-a3b9-87f5ef7a268a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:29.767348 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc08f67-5dd4-44d3-a3b9-87f5ef7a268a None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:29.782532 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-5bc08f67-5dd4-44d3-a3b9-87f5ef7a268a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:29.782532 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:29.782532 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:29.782532 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:29.782532 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:29.782532 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:29.782532 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:29.782532 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:29.782532 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:29.782532 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:29.782532 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:29.782532 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:29.782532 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 21:50:29.782532 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 21:50:29.782532 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 21:50:29.784100 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 21:50:29.784100 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:29.784100 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:29.784100 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:29.784100 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:29.784100 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:29.784100 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:29.784100 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:29.784100 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:29.784100 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:29.784100 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:29.784100 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:29.784100 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:29.784100 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:29.784100 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:29.784100 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:29.784100 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:29.785230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 21:50:29.785230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 21:50:29.785230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 21:50:29.785230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 21:50:29.785230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 21:50:29.785230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 21:50:29.785230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:29.785230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:29.785230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 388/767] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:50:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:29.786683 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc08f67-5dd4-44d3-a3b9-87f5ef7a268a None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:29.809469 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a9ad15-5413-4e55-be35-3bd84ac2abcc None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:29.825384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:29.850125 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23db56b-6427-4972-81e3-d5fb785a1bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:29.851040 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23db56b-6427-4972-81e3-d5fb785a1bb6 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:29.851524 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23db56b-6427-4972-81e3-d5fb785a1bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:29.852003 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23db56b-6427-4972-81e3-d5fb785a1bb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:29.853070 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23db56b-6427-4972-81e3-d5fb785a1bb6 None admin] RBAC: Adding query filter params (role.id=2c7f73c184d0406bbf2899465953342b, scope.project.id=0219dbda54274fdc9bdccbd960c30c6b, user.id=86619a80df8841a3ba7d8471ac38056a) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:29.853549 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23db56b-6427-4972-81e3-d5fb785a1bb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:29.855167 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23db56b-6427-4972-81e3-d5fb785a1bb6 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:29.862274 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 380/768] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:50:29 2022] GET /identity/v3/role_assignments?user.id=86619a80df8841a3ba7d8471ac38056a&scope.project.id=0219dbda54274fdc9bdccbd960c30c6b&role.id=2c7f73c184d0406bbf2899465953342b => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:29.863268 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23db56b-6427-4972-81e3-d5fb785a1bb6 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:29.898946 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:29.917650 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc08f67-5dd4-44d3-a3b9-87f5ef7a268a None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:29.980712 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23db56b-6427-4972-81e3-d5fb785a1bb6 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:29.989244 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99087e8d-705a-4c4f-81cb-963a85470a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:29.990285 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99087e8d-705a-4c4f-81cb-963a85470a9d None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:29.990285 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99087e8d-705a-4c4f-81cb-963a85470a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:29.990514 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99087e8d-705a-4c4f-81cb-963a85470a9d None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:29.991162 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99087e8d-705a-4c4f-81cb-963a85470a9d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:29.991493 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99087e8d-705a-4c4f-81cb-963a85470a9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:29.993090 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99087e8d-705a-4c4f-81cb-963a85470a9d None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:30.007094 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 389/769] 10.222.0.13 () {56 vars in 1124 bytes} [Tue Jun 7 21:50:29 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:30.012720 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99087e8d-705a-4c4f-81cb-963a85470a9d None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:30.022385 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0e9929-7a71-4164-9968-319ad158685a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:30.023374 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0e9929-7a71-4164-9968-319ad158685a None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:30.023798 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0e9929-7a71-4164-9968-319ad158685a None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:30.024211 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0e9929-7a71-4164-9968-319ad158685a None admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:30.025290 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0e9929-7a71-4164-9968-319ad158685a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:30.025743 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0e9929-7a71-4164-9968-319ad158685a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:30.027342 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0e9929-7a71-4164-9968-319ad158685a None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:30.041687 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 381/770] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jun 7 21:50:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:30.047517 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0e9929-7a71-4164-9968-319ad158685a None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:30.146873 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:30.219782 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99087e8d-705a-4c4f-81cb-963a85470a9d None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:30.261287 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0e9929-7a71-4164-9968-319ad158685a None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:30.286404 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7449ee6-4086-478c-bc87-6170cd00abd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:30.287076 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7449ee6-4086-478c-bc87-6170cd00abd5 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:30.287380 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7449ee6-4086-478c-bc87-6170cd00abd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:30.287669 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7449ee6-4086-478c-bc87-6170cd00abd5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:30.288470 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7449ee6-4086-478c-bc87-6170cd00abd5 None admin] RBAC: Adding query filter params (interface=public, service_id=d47ac3e4566b434b9da408e1a13c6208, region_id=RegionOne) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:30.288784 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7449ee6-4086-478c-bc87-6170cd00abd5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:30.290165 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7449ee6-4086-478c-bc87-6170cd00abd5 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:30.305950 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 390/771] 10.222.0.13 () {56 vars in 1251 bytes} [Tue Jun 7 21:50:30 2022] GET /identity/v3/endpoints?service_id=d47ac3e4566b434b9da408e1a13c6208&interface=public®ion_id=RegionOne => generated 194 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:50:30.309782 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7449ee6-4086-478c-bc87-6170cd00abd5 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:30.328544 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8345d0c0-5668-4f4c-a352-74892430f55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:30.329205 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8345d0c0-5668-4f4c-a352-74892430f55a None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:30.329491 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8345d0c0-5668-4f4c-a352-74892430f55a None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:30.329771 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8345d0c0-5668-4f4c-a352-74892430f55a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:30.330754 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8345d0c0-5668-4f4c-a352-74892430f55a None admin] RBAC: Adding query filter params (role.id=6b483758dbd44eb4a78df2ce4603d601, scope.project.id=0219dbda54274fdc9bdccbd960c30c6b, user.id=7a1261563b6d4e50a4ffeea6a32bdb01) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:30.331070 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8345d0c0-5668-4f4c-a352-74892430f55a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:30.332539 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8345d0c0-5668-4f4c-a352-74892430f55a None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:30.348984 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 382/772] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:50:30 2022] GET /identity/v3/role_assignments?user.id=7a1261563b6d4e50a4ffeea6a32bdb01&scope.project.id=0219dbda54274fdc9bdccbd960c30c6b&role.id=6b483758dbd44eb4a78df2ce4603d601 => generated 259 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:50:30.359193 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:30.475152 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7449ee6-4086-478c-bc87-6170cd00abd5 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:30.527196 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e297a3c-db2a-4a3d-8166-7d3b680608f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:30.527931 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e297a3c-db2a-4a3d-8166-7d3b680608f9 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:30.528232 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e297a3c-db2a-4a3d-8166-7d3b680608f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:30.528496 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e297a3c-db2a-4a3d-8166-7d3b680608f9 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:30.529207 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e297a3c-db2a-4a3d-8166-7d3b680608f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:30.530620 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e297a3c-db2a-4a3d-8166-7d3b680608f9 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:30.537761 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 391/773] 10.222.0.13 () {56 vars in 1087 bytes} [Tue Jun 7 21:50:30 2022] GET /identity/v3/services => generated 772 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:30.571614 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18ddbd1-3272-4f53-8556-b59a5f3c1ac2 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:30.572300 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18ddbd1-3272-4f53-8556-b59a5f3c1ac2 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:30.572795 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18ddbd1-3272-4f53-8556-b59a5f3c1ac2 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:30.574867 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 383/774] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:30.581512 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66dd57-daf7-447b-9eb8-dbe099693531 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:30.582157 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66dd57-daf7-447b-9eb8-dbe099693531 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:30.582640 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66dd57-daf7-447b-9eb8-dbe099693531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:30.602028 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-ee66dd57-daf7-447b-9eb8-dbe099693531 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:30.603630 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-ee66dd57-daf7-447b-9eb8-dbe099693531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:30.697393 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 392/775] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:50:30.724910 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18ddbd1-3272-4f53-8556-b59a5f3c1ac2 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:30.843656 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18ddbd1-3272-4f53-8556-b59a5f3c1ac2 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:30.845667 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:30.883054 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2e721c-4f11-49d0-ae0a-28936cbc0c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:30.884252 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e721c-4f11-49d0-ae0a-28936cbc0c4e None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:30.884733 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e721c-4f11-49d0-ae0a-28936cbc0c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:30.885229 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e721c-4f11-49d0-ae0a-28936cbc0c4e None admin] PATH_INFO: `/v3/services/network` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:30.886083 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2e721c-4f11-49d0-ae0a-28936cbc0c4e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:30.887726 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2e721c-4f11-49d0-ae0a-28936cbc0c4e None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:30.901333 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-be2e721c-4f11-49d0-ae0a-28936cbc0c4e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 21:50:30.901333 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:30.901333 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 21:50:30.901333 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 21:50:30.901333 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 21:50:30.901333 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 21:50:30.901333 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 21:50:30.901333 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:50:30.901333 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 21:50:30.901333 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:30.901333 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 21:50:30.901333 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:30.901333 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:30.901333 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:30.901333 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:30.901333 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:30.901333 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:30.902368 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:30.902368 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:30.902368 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:30.902368 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:30.902368 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:30.902368 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:30.902368 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 21:50:30.902368 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 21:50:30.902368 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 21:50:30.902368 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 21:50:30.902368 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:30.902368 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:30.902368 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:30.902368 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:30.902368 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:30.902368 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:30.903205 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:30.903205 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:30.903205 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:30.903205 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:30.903205 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:30.903205 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:30.903205 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:30.903205 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:30.903205 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:30.903205 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:30.903205 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 21:50:30.903205 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:50:30.903205 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 21:50:30.903205 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:30.905645 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 384/776] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:50:30 2022] GET /identity/v3/services/network => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:30.911877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee66dd57-daf7-447b-9eb8-dbe099693531 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:30.920163 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761bade3-d5dc-4020-b11a-0f09a83359c3 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:30.920455 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761bade3-d5dc-4020-b11a-0f09a83359c3 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:30.920738 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761bade3-d5dc-4020-b11a-0f09a83359c3 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:30.922420 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 385/777] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:30.928405 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469b59a-e0e7-499f-91cb-3bcb5f611b1b None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:30.928698 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469b59a-e0e7-499f-91cb-3bcb5f611b1b None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:30.928982 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469b59a-e0e7-499f-91cb-3bcb5f611b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:30.932694 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:30.952696 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-8469b59a-e0e7-499f-91cb-3bcb5f611b1b None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:30.953906 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-8469b59a-e0e7-499f-91cb-3bcb5f611b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:31.035742 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 386/778] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 21:50:31.058975 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469b59a-e0e7-499f-91cb-3bcb5f611b1b None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:31.072670 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee66dd57-daf7-447b-9eb8-dbe099693531 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:31.161836 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b307a5de-a1e5-4ff0-91f5-c4b489d5979c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:31.162502 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307a5de-a1e5-4ff0-91f5-c4b489d5979c None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:31.162779 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307a5de-a1e5-4ff0-91f5-c4b489d5979c None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:31.163048 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307a5de-a1e5-4ff0-91f5-c4b489d5979c None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:31.163787 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b307a5de-a1e5-4ff0-91f5-c4b489d5979c None admin] RBAC: Adding query filter params (name=network) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:31.164090 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b307a5de-a1e5-4ff0-91f5-c4b489d5979c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:31.165699 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b307a5de-a1e5-4ff0-91f5-c4b489d5979c None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:31.176216 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 393/779] 10.222.0.13 () {56 vars in 1112 bytes} [Tue Jun 7 21:50:30 2022] GET /identity/v3/services?name=network => generated 124 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:31.181813 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b307a5de-a1e5-4ff0-91f5-c4b489d5979c None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:31.199435 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:31.209653 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469b59a-e0e7-499f-91cb-3bcb5f611b1b None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:31.267707 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab21602-2b43-49a9-947e-fd9da562e23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:31.268783 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab21602-2b43-49a9-947e-fd9da562e23b None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:31.269062 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab21602-2b43-49a9-947e-fd9da562e23b None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:31.269333 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab21602-2b43-49a9-947e-fd9da562e23b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:31.292259 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab21602-2b43-49a9-947e-fd9da562e23b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:50:31.306134 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-5ab21602-2b43-49a9-947e-fd9da562e23b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:50:31.306134 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:31.306134 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:31.306134 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:31.306134 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:31.306134 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:31.306134 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:31.306134 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:31.306134 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:31.306134 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:31.306134 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:31.306134 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:31.306134 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:50:31.306134 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:50:31.306134 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:50:31.306134 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:50:31.307083 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:50:31.307083 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:50:31.307083 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:50:31.307083 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:50:31.307083 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:31.307083 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:31.307083 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:31.307083 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:31.307083 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:31.307083 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:31.307083 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:31.307083 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:31.307083 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:31.307083 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:31.307083 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:31.307083 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:31.307962 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:31.307962 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:31.307962 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:31.307962 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:31.307962 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:50:31.307962 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:50:31.307962 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:50:31.307962 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:50:31.307962 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:50:31.307962 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:50:31.307962 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:50:31.307962 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:31.309090 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 387/780] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jun 7 21:50:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:50:31.313404 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab21602-2b43-49a9-947e-fd9da562e23b None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:31.330140 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:31.354339 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b307a5de-a1e5-4ff0-91f5-c4b489d5979c None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:31.394250 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d04c1-5202-4884-9633-960d22556233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:31.394250 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d04c1-5202-4884-9633-960d22556233 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:31.394250 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d04c1-5202-4884-9633-960d22556233 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:31.394250 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d04c1-5202-4884-9633-960d22556233 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:31.394686 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189d04c1-5202-4884-9633-960d22556233 None admin] RBAC: Adding query filter params (type=network) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:31.394686 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189d04c1-5202-4884-9633-960d22556233 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:31.395618 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189d04c1-5202-4884-9633-960d22556233 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:31.404055 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 394/781] 10.222.0.13 () {56 vars in 1112 bytes} [Tue Jun 7 21:50:31 2022] GET /identity/v3/services?type=network => generated 350 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:31.410513 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d04c1-5202-4884-9633-960d22556233 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:31.516351 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d04c1-5202-4884-9633-960d22556233 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:31.551947 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8022b23-3ac5-48df-ac98-f3e471f0a8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:31.552523 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8022b23-3ac5-48df-ac98-f3e471f0a8a0 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:31.552949 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8022b23-3ac5-48df-ac98-f3e471f0a8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:31.553521 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8022b23-3ac5-48df-ac98-f3e471f0a8a0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:31.554044 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8022b23-3ac5-48df-ac98-f3e471f0a8a0 None admin] RBAC: Adding query filter params (interface=public, service_id=ee3a68fb020c4f36831a11e8c6b63122, region_id=RegionOne) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:31.554355 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8022b23-3ac5-48df-ac98-f3e471f0a8a0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:31.555893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8022b23-3ac5-48df-ac98-f3e471f0a8a0 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:31.561284 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 395/782] 10.222.0.13 () {56 vars in 1251 bytes} [Tue Jun 7 21:50:31 2022] GET /identity/v3/endpoints?service_id=ee3a68fb020c4f36831a11e8c6b63122&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:50:31.566909 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8022b23-3ac5-48df-ac98-f3e471f0a8a0 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:31.577966 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:31.629527 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab21602-2b43-49a9-947e-fd9da562e23b None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:31.675175 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8022b23-3ac5-48df-ac98-f3e471f0a8a0 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:31.711543 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bad086f-a088-4799-896b-ef4e3d03c098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:31.712221 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bad086f-a088-4799-896b-ef4e3d03c098 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:31.712500 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bad086f-a088-4799-896b-ef4e3d03c098 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:31.712776 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bad086f-a088-4799-896b-ef4e3d03c098 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:31.713455 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bad086f-a088-4799-896b-ef4e3d03c098 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:31.714814 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bad086f-a088-4799-896b-ef4e3d03c098 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:31.721003 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 396/783] 10.222.0.13 () {56 vars in 1087 bytes} [Tue Jun 7 21:50:31 2022] GET /identity/v3/services => generated 772 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:50:31.723993 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06aad75-41ec-4090-a56c-6ca3781f17e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:31.724683 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06aad75-41ec-4090-a56c-6ca3781f17e9 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:31.724965 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06aad75-41ec-4090-a56c-6ca3781f17e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:31.725242 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06aad75-41ec-4090-a56c-6ca3781f17e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:31.726021 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06aad75-41ec-4090-a56c-6ca3781f17e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:31.726331 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06aad75-41ec-4090-a56c-6ca3781f17e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:31.727794 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06aad75-41ec-4090-a56c-6ca3781f17e9 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:31.746706 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 388/784] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:50:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 434 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:50:31.753435 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bad086f-a088-4799-896b-ef4e3d03c098 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:31.885068 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:31.931638 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bad086f-a088-4799-896b-ef4e3d03c098 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:31.974286 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b6b0b-0773-4cea-a716-1f8d709eb1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:31.975087 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b6b0b-0773-4cea-a716-1f8d709eb1da None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:31.975397 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b6b0b-0773-4cea-a716-1f8d709eb1da None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:31.975675 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b6b0b-0773-4cea-a716-1f8d709eb1da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:31.986809 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4b6b0b-0773-4cea-a716-1f8d709eb1da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:31.988446 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4b6b0b-0773-4cea-a716-1f8d709eb1da None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:32.006566 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-ae4b6b0b-0773-4cea-a716-1f8d709eb1da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:32.006566 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:32.006566 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:32.006566 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:32.006566 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:32.006566 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:32.006566 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:32.006566 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:32.006566 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:32.006566 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:32.006566 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:32.006566 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:32.006566 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:32.006566 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:32.006566 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:32.007536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:32.007536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:32.007536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:32.007536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:32.007536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:32.007536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:32.007536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:32.007536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:32.007536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:32.007536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:32.007536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:32.007536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:32.007536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:32.007536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:32.007536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:32.007536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:32.007536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:32.008425 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:32.008425 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:32.008425 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:32.008425 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:32.008425 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 397/785] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:32.013969 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06aad75-41ec-4090-a56c-6ca3781f17e9 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:32.035773 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:32.131476 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06aad75-41ec-4090-a56c-6ca3781f17e9 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:32.176113 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efac210-1ac4-46bd-aa3b-f64188b3b7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:32.176699 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efac210-1ac4-46bd-aa3b-f64188b3b7a2 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:32.176979 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efac210-1ac4-46bd-aa3b-f64188b3b7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:32.177252 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efac210-1ac4-46bd-aa3b-f64188b3b7a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:32.177973 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efac210-1ac4-46bd-aa3b-f64188b3b7a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:32.178276 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efac210-1ac4-46bd-aa3b-f64188b3b7a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:32.179671 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efac210-1ac4-46bd-aa3b-f64188b3b7a2 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:32.193244 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 389/786] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:32.198484 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b6b0b-0773-4cea-a716-1f8d709eb1da None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:32.325469 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b6b0b-0773-4cea-a716-1f8d709eb1da None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:32.354209 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:32.369698 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9aa145-500c-43fa-879d-9adeb9ba666c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:32.372211 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9aa145-500c-43fa-879d-9adeb9ba666c None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:32.372506 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9aa145-500c-43fa-879d-9adeb9ba666c None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:32.375700 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9aa145-500c-43fa-879d-9adeb9ba666c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:32.385011 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9aa145-500c-43fa-879d-9adeb9ba666c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:32.386467 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9aa145-500c-43fa-879d-9adeb9ba666c None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:32.405004 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-3e9aa145-500c-43fa-879d-9adeb9ba666c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 21:50:32.405004 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:32.405004 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:32.405004 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:32.405004 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:32.405004 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:32.405004 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:32.405004 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:32.405004 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:32.405004 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:32.405004 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:32.405004 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:32.405004 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 21:50:32.405004 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 21:50:32.405004 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 21:50:32.405004 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 21:50:32.406006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:32.406006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:32.406006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 21:50:32.406006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:32.406006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 21:50:32.406006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:32.406006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:32.406006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:32.406006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:32.406006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:32.406006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:32.406006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:32.406006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:32.406006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:32.406006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:32.406006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:32.406006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:32.406006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:32.406985 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:32.406985 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:32.406985 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 21:50:32.406985 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 21:50:32.406985 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 21:50:32.406985 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 21:50:32.406985 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 21:50:32.406985 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 21:50:32.406985 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 21:50:32.406985 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:32.406985 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 398/787] 10.222.0.13 () {56 vars in 1093 bytes} [Tue Jun 7 21:50:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:32.412990 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efac210-1ac4-46bd-aa3b-f64188b3b7a2 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:32.527872 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:32.561490 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efac210-1ac4-46bd-aa3b-f64188b3b7a2 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:32.606896 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33310fe5-7e63-4df8-85b9-ed71c06279db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:32.607614 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33310fe5-7e63-4df8-85b9-ed71c06279db None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:32.607902 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33310fe5-7e63-4df8-85b9-ed71c06279db None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:32.608181 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33310fe5-7e63-4df8-85b9-ed71c06279db None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:32.609027 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33310fe5-7e63-4df8-85b9-ed71c06279db None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:32.609339 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33310fe5-7e63-4df8-85b9-ed71c06279db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:32.611002 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33310fe5-7e63-4df8-85b9-ed71c06279db None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:32.639714 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 390/788] 10.222.0.13 () {56 vars in 1133 bytes} [Tue Jun 7 21:50:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:32.645826 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9aa145-500c-43fa-879d-9adeb9ba666c None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:32.753620 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9aa145-500c-43fa-879d-9adeb9ba666c None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:32.789897 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d86a844-9f91-4be3-84aa-feb1baaa276c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:32.790558 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86a844-9f91-4be3-84aa-feb1baaa276c None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:32.790842 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86a844-9f91-4be3-84aa-feb1baaa276c None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:32.791118 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86a844-9f91-4be3-84aa-feb1baaa276c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:32.859051 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:32.883920 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d86a844-9f91-4be3-84aa-feb1baaa276c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:32.886244 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d86a844-9f91-4be3-84aa-feb1baaa276c None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:32.913121 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a54abc-4d68-46ee-9bbe-e481ce1b4a9b None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:32.913449 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a54abc-4d68-46ee-9bbe-e481ce1b4a9b None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:32.913671 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a54abc-4d68-46ee-9bbe-e481ce1b4a9b None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:32.919408 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 391/789] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 21:50:32.920936 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4079b19-0f3f-41ed-b5ed-9ebd390e4a73 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:32.921227 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4079b19-0f3f-41ed-b5ed-9ebd390e4a73 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:32.921503 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4079b19-0f3f-41ed-b5ed-9ebd390e4a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:32.939850 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-d4079b19-0f3f-41ed-b5ed-9ebd390e4a73 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:32.941463 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-d4079b19-0f3f-41ed-b5ed-9ebd390e4a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:33.004144 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 399/790] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:50:32 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 21:50:33.013440 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d86a844-9f91-4be3-84aa-feb1baaa276c None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:33.065112 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 392/791] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:50:33.104071 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4079b19-0f3f-41ed-b5ed-9ebd390e4a73 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:33.130145 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:33.159074 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d86a844-9f91-4be3-84aa-feb1baaa276c None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:33.241665 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9421402-2df4-48aa-8e77-e9823b297fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:33.242473 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9421402-2df4-48aa-8e77-e9823b297fd5 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:33.242910 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9421402-2df4-48aa-8e77-e9823b297fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:33.243194 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9421402-2df4-48aa-8e77-e9823b297fd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:33.255357 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9421402-2df4-48aa-8e77-e9823b297fd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:33.256840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9421402-2df4-48aa-8e77-e9823b297fd5 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:33.279051 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-b9421402-2df4-48aa-8e77-e9823b297fd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:33.279051 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:33.279051 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:33.279051 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:33.279051 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:33.279051 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:33.279051 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:33.279051 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:33.279051 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:33.279051 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:33.279051 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:33.279051 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:33.279051 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:33.279051 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:33.279051 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:33.279999 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:33.279999 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:33.279999 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:33.279999 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:33.279999 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:33.279999 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:33.279999 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:33.279999 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:33.279999 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:33.279999 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:33.279999 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:33.279999 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:33.279999 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:33.279999 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:33.279999 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:33.279999 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:33.279999 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:33.280891 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:33.280891 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:33.280891 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:33.280891 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:33.283018 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:33.292554 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 400/792] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:33.296675 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9421402-2df4-48aa-8e77-e9823b297fd5 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:33.355658 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4079b19-0f3f-41ed-b5ed-9ebd390e4a73 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:33.397523 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b79a14-ac37-4d7d-959c-f36ee89b5349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:33.398313 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b79a14-ac37-4d7d-959c-f36ee89b5349 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:33.398593 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b79a14-ac37-4d7d-959c-f36ee89b5349 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:33.398857 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b79a14-ac37-4d7d-959c-f36ee89b5349 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:33.410140 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b79a14-ac37-4d7d-959c-f36ee89b5349 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:50:33.417813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-88b79a14-ac37-4d7d-959c-f36ee89b5349 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 21:50:33.417813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:33.417813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:33.417813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:33.417813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:33.417813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:33.417813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:33.417813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:33.417813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:33.417813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:33.417813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:33.417813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:33.417813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:50:33.417813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:50:33.417813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:50:33.417813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:50:33.419982 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:50:33.419982 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:50:33.419982 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:50:33.419982 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:50:33.419982 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:33.419982 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:33.419982 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:33.419982 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:33.419982 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:33.419982 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:33.419982 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:33.419982 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:33.419982 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:33.419982 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:33.419982 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:33.419982 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:33.422079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:33.422079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:33.422079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:33.422079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:33.422079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:50:33.422079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:50:33.422079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:50:33.422079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:50:33.422079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:50:33.422079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:50:33.422079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 21:50:33.422079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:33.422079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 393/793] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jun 7 21:50:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:50:33.422079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb6e0c-7f5f-4c64-ba49-900ec0553055 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:33.431970 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb6e0c-7f5f-4c64-ba49-900ec0553055 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:33.431970 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb6e0c-7f5f-4c64-ba49-900ec0553055 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:33.431970 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 394/794] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:33.431970 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb6e0c-7f5f-4c64-ba49-900ec0553055 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:33.447267 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9421402-2df4-48aa-8e77-e9823b297fd5 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:33.491552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bcef6e-95b5-401a-8212-9a0902b279c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:33.492530 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bcef6e-95b5-401a-8212-9a0902b279c6 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:33.492964 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bcef6e-95b5-401a-8212-9a0902b279c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:33.493433 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bcef6e-95b5-401a-8212-9a0902b279c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:33.494773 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bcef6e-95b5-401a-8212-9a0902b279c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:33.495263 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bcef6e-95b5-401a-8212-9a0902b279c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:33.497217 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bcef6e-95b5-401a-8212-9a0902b279c6 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:33.501237 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:33.515778 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 401/795] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:33.517934 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb8d14-745b-40a9-ae5e-50e968ad2770 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:33.518507 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb8d14-745b-40a9-ae5e-50e968ad2770 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:33.518987 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb8d14-745b-40a9-ae5e-50e968ad2770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:33.544590 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-33bb8d14-745b-40a9-ae5e-50e968ad2770 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:33.547493 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-33bb8d14-745b-40a9-ae5e-50e968ad2770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:33.574290 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb6e0c-7f5f-4c64-ba49-900ec0553055 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:33.603662 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:33.614525 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fdde92-74fb-4de7-8b46-1a6363648dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:33.615129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdde92-74fb-4de7-8b46-1a6363648dc2 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:33.615431 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdde92-74fb-4de7-8b46-1a6363648dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:33.615705 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdde92-74fb-4de7-8b46-1a6363648dc2 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:33.616599 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fdde92-74fb-4de7-8b46-1a6363648dc2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:33.616732 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fdde92-74fb-4de7-8b46-1a6363648dc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:33.618084 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fdde92-74fb-4de7-8b46-1a6363648dc2 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:33.627488 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 402/796] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:50:33.627488 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb8d14-745b-40a9-ae5e-50e968ad2770 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:33.634261 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 395/797] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Jun 7 21:50:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:33.634261 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cc525-d513-43a0-8a2d-22f50b2514d2 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:33.634261 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cc525-d513-43a0-8a2d-22f50b2514d2 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:33.634261 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cc525-d513-43a0-8a2d-22f50b2514d2 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:33.634261 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 396/798] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:33.636525 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1cc525-d513-43a0-8a2d-22f50b2514d2 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:33.741229 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb8d14-745b-40a9-ae5e-50e968ad2770 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:33.748169 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1cc525-d513-43a0-8a2d-22f50b2514d2 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:33.773098 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:33.782960 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a847cc2b-280b-4639-a5af-45306f279f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:33.784045 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a847cc2b-280b-4639-a5af-45306f279f71 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:33.784326 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a847cc2b-280b-4639-a5af-45306f279f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:33.784600 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a847cc2b-280b-4639-a5af-45306f279f71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:33.789308 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbbee18-00e7-41f0-be96-acabea366cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:33.790344 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbbee18-00e7-41f0-be96-acabea366cd5 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:33.790830 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbbee18-00e7-41f0-be96-acabea366cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:33.791312 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbbee18-00e7-41f0-be96-acabea366cd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:33.791723 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a847cc2b-280b-4639-a5af-45306f279f71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:33.793166 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a847cc2b-280b-4639-a5af-45306f279f71 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:33.803316 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbbee18-00e7-41f0-be96-acabea366cd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:33.805144 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbbee18-00e7-41f0-be96-acabea366cd5 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:33.807726 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-a847cc2b-280b-4639-a5af-45306f279f71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:33.807726 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:33.807726 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:33.807726 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:33.807726 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:33.807726 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:33.807726 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:33.807726 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:33.807726 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:33.807726 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:33.807726 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:33.807726 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:33.807726 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 21:50:33.807726 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 21:50:33.807726 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 21:50:33.808577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 21:50:33.808577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:33.808577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:33.808577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:33.808577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:33.808577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:33.808577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:33.808577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:33.808577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:33.808577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:33.808577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:33.808577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:33.808577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:33.808577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:33.808577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:33.808577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:33.808577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:33.809475 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 21:50:33.809475 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 21:50:33.809475 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 21:50:33.809475 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 21:50:33.809475 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 21:50:33.809475 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 21:50:33.809475 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:33.809475 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:33.809475 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 403/799] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:50:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:33.810247 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a87fc0-6017-4186-8b07-0e1705e9122c None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:33.810344 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a87fc0-6017-4186-8b07-0e1705e9122c None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:33.810590 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a87fc0-6017-4186-8b07-0e1705e9122c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:33.827848 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-2fbbee18-00e7-41f0-be96-acabea366cd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:33.827848 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:33.827848 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:33.827848 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:33.827848 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:33.827848 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:33.827848 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:33.827848 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:33.827848 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:33.827848 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:33.827848 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:33.827848 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:33.827848 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:33.827848 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:33.827848 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:33.828769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:33.828769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:33.828769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:33.828769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:33.828769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:33.828769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:33.828769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:33.828769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:33.828769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:33.828769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:33.828769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:33.828769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:33.828769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:33.828769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:33.828769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:33.828769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:33.828769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:33.830271 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:33.830271 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:33.830271 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:33.830271 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:33.831868 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-19a87fc0-6017-4186-8b07-0e1705e9122c None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:33.832346 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 397/800] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:50:33.833253 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbbee18-00e7-41f0-be96-acabea366cd5 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:33.837519 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-19a87fc0-6017-4186-8b07-0e1705e9122c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:33.882599 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:33.910544 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 404/801] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:50:33.911522 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a87fc0-6017-4186-8b07-0e1705e9122c None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:33.953559 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbbee18-00e7-41f0-be96-acabea366cd5 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:33.988896 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392b4e48-09eb-4d8a-b677-f6f2794b47a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:33.989693 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b4e48-09eb-4d8a-b677-f6f2794b47a0 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:33.989978 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b4e48-09eb-4d8a-b677-f6f2794b47a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:33.990255 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b4e48-09eb-4d8a-b677-f6f2794b47a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:34.000459 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392b4e48-09eb-4d8a-b677-f6f2794b47a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:50:34.008126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-392b4e48-09eb-4d8a-b677-f6f2794b47a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:50:34.008126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:34.008126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:34.008126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:34.008126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:34.008126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:34.008126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:34.008126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:34.008126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:34.008126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:34.008126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:34.008126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:34.008126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:50:34.008126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:50:34.008126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:50:34.008126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:50:34.009063 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:50:34.009063 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:50:34.009063 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:50:34.009063 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:50:34.009063 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:34.009063 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:34.009063 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:34.009063 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:34.009063 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:34.009063 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:34.009063 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:34.009063 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:34.009063 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:34.009063 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:34.009063 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:34.009063 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:34.009920 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:34.009920 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:34.009920 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:34.009920 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:34.009920 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:50:34.009920 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:50:34.009920 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:50:34.009920 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:50:34.009920 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:50:34.009920 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:50:34.009920 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:50:34.009920 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:34.009920 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 398/802] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jun 7 21:50:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:34.010618 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392b4e48-09eb-4d8a-b677-f6f2794b47a0 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:34.027185 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a87fc0-6017-4186-8b07-0e1705e9122c None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:34.045987 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:34.064796 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8991e9f8-3154-4b0a-bc81-311ad2b4bab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:34.065513 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991e9f8-3154-4b0a-bc81-311ad2b4bab1 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:34.065740 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991e9f8-3154-4b0a-bc81-311ad2b4bab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:34.066014 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991e9f8-3154-4b0a-bc81-311ad2b4bab1 None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:34.066807 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8991e9f8-3154-4b0a-bc81-311ad2b4bab1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:34.067107 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8991e9f8-3154-4b0a-bc81-311ad2b4bab1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:34.068564 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8991e9f8-3154-4b0a-bc81-311ad2b4bab1 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:34.080217 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 405/803] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jun 7 21:50:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:34.081028 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8991e9f8-3154-4b0a-bc81-311ad2b4bab1 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:34.130203 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:34.133996 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392b4e48-09eb-4d8a-b677-f6f2794b47a0 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:34.170427 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2f88de-9e7b-4107-842c-2b0688e405be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:34.171539 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f88de-9e7b-4107-842c-2b0688e405be None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:34.172046 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f88de-9e7b-4107-842c-2b0688e405be None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:34.172547 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f88de-9e7b-4107-842c-2b0688e405be None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:34.174104 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2f88de-9e7b-4107-842c-2b0688e405be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:34.174651 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2f88de-9e7b-4107-842c-2b0688e405be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:34.176099 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2f88de-9e7b-4107-842c-2b0688e405be None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:34.190395 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 399/804] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:34.191097 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2f88de-9e7b-4107-842c-2b0688e405be None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:34.200408 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8991e9f8-3154-4b0a-bc81-311ad2b4bab1 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:34.238543 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fc760b-3051-4334-b7ea-b71687323be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:34.239398 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fc760b-3051-4334-b7ea-b71687323be3 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:34.239652 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fc760b-3051-4334-b7ea-b71687323be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:34.239920 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fc760b-3051-4334-b7ea-b71687323be3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:34.240567 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fc760b-3051-4334-b7ea-b71687323be3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:34.241923 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fc760b-3051-4334-b7ea-b71687323be3 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:34.253976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-b6fc760b-3051-4334-b7ea-b71687323be3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 21:50:34.253976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:34.253976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 21:50:34.253976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 21:50:34.253976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 21:50:34.253976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 21:50:34.253976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 21:50:34.253976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:50:34.253976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 21:50:34.253976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:34.253976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 21:50:34.253976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:34.253976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:34.253976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:34.253976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:34.253976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:34.254934 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:34.254934 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:34.254934 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:34.254934 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:34.254934 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:34.254934 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:34.254934 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:34.254934 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 21:50:34.254934 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 21:50:34.254934 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 21:50:34.254934 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 21:50:34.254934 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:34.254934 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:34.254934 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:34.254934 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:34.254934 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:34.255841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:34.255841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:34.255841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:34.255841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:34.255841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:34.255841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:34.255841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:34.255841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:34.255841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:34.255841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:34.255841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:34.255841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 21:50:34.255841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:50:34.255841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 21:50:34.255841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:34.255841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 406/805] 10.222.0.13 () {56 vars in 1129 bytes} [Tue Jun 7 21:50:34 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:34.257506 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fc760b-3051-4334-b7ea-b71687323be3 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:34.304267 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:34.305267 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2f88de-9e7b-4107-842c-2b0688e405be None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:34.342241 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a61a6-58d7-4791-a071-42a53d8f5f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:34.343016 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a61a6-58d7-4791-a071-42a53d8f5f5f None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:34.343289 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a61a6-58d7-4791-a071-42a53d8f5f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:34.343605 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a61a6-58d7-4791-a071-42a53d8f5f5f None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:34.344357 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950a61a6-58d7-4791-a071-42a53d8f5f5f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:34.344664 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950a61a6-58d7-4791-a071-42a53d8f5f5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:34.346703 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950a61a6-58d7-4791-a071-42a53d8f5f5f None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:34.355354 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 400/806] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:50:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:34.356166 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a61a6-58d7-4791-a071-42a53d8f5f5f None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:34.382092 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fc760b-3051-4334-b7ea-b71687323be3 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:34.384119 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:34.420372 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17621a-ba63-4c51-ae46-4ca1494370ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:34.421111 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17621a-ba63-4c51-ae46-4ca1494370ae None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:34.421323 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17621a-ba63-4c51-ae46-4ca1494370ae None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:34.421589 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17621a-ba63-4c51-ae46-4ca1494370ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:34.422477 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b17621a-ba63-4c51-ae46-4ca1494370ae None admin] RBAC: Adding query filter params (role.id=6b483758dbd44eb4a78df2ce4603d601, scope.project.id=0219dbda54274fdc9bdccbd960c30c6b, user.id=dbe0303ea0764347bfb054870ebba41f) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:34.422717 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b17621a-ba63-4c51-ae46-4ca1494370ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:34.424199 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b17621a-ba63-4c51-ae46-4ca1494370ae None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:34.439381 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 407/807] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:50:34 2022] GET /identity/v3/role_assignments?user.id=dbe0303ea0764347bfb054870ebba41f&scope.project.id=0219dbda54274fdc9bdccbd960c30c6b&role.id=6b483758dbd44eb4a78df2ce4603d601 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:34.439381 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17621a-ba63-4c51-ae46-4ca1494370ae None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:34.481093 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a61a6-58d7-4791-a071-42a53d8f5f5f None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:34.528382 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80566542-704f-410b-afb3-59fd50c20671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:34.529222 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80566542-704f-410b-afb3-59fd50c20671 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:34.529506 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80566542-704f-410b-afb3-59fd50c20671 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:34.529786 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80566542-704f-410b-afb3-59fd50c20671 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:34.539591 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80566542-704f-410b-afb3-59fd50c20671 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:34.541537 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80566542-704f-410b-afb3-59fd50c20671 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:34.560158 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-80566542-704f-410b-afb3-59fd50c20671 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 21:50:34.560158 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:34.560158 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:34.560158 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:34.560158 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:34.560158 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:34.560158 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:34.560158 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:34.560158 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:34.560158 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:34.560158 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:34.560158 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:34.560158 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 21:50:34.560158 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 21:50:34.560158 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 21:50:34.560158 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 21:50:34.561221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:34.561221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:34.561221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 21:50:34.561221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:34.561221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 21:50:34.561221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:34.561221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:34.561221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:34.561221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:34.561221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:34.561221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:34.561221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:34.561221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:34.561221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:34.561221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:34.561221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:34.561221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:34.561221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:34.562161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:34.562161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:34.562161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 21:50:34.562161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 21:50:34.562161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 21:50:34.562161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 21:50:34.562161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 21:50:34.562161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 21:50:34.562161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 21:50:34.562161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:34.562161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 401/808] 10.222.0.13 () {56 vars in 1117 bytes} [Tue Jun 7 21:50:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:34.562772 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80566542-704f-410b-afb3-59fd50c20671 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:34.574620 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17621a-ba63-4c51-ae46-4ca1494370ae None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:34.578770 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:34.617969 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184cf527-1c4c-46df-9d43-776a44391340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:34.618656 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184cf527-1c4c-46df-9d43-776a44391340 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:34.618937 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184cf527-1c4c-46df-9d43-776a44391340 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:34.619215 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184cf527-1c4c-46df-9d43-776a44391340 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:34.619971 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184cf527-1c4c-46df-9d43-776a44391340 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:34.620281 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184cf527-1c4c-46df-9d43-776a44391340 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:34.622097 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184cf527-1c4c-46df-9d43-776a44391340 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:34.632231 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 408/809] 10.222.0.13 () {56 vars in 1124 bytes} [Tue Jun 7 21:50:34 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:34.633561 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c6fbc-6cf7-48aa-b7be-ce2782df607c None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:34.633853 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c6fbc-6cf7-48aa-b7be-ce2782df607c None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:34.634184 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c6fbc-6cf7-48aa-b7be-ce2782df607c None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:34.639530 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 409/810] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:34.639530 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c6fbc-6cf7-48aa-b7be-ce2782df607c None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:34.660795 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:34.682009 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80566542-704f-410b-afb3-59fd50c20671 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:34.718479 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c39273-ceca-4fe8-badb-d603063a19c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:34.719272 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c39273-ceca-4fe8-badb-d603063a19c5 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:34.719595 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c39273-ceca-4fe8-badb-d603063a19c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:34.719877 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c39273-ceca-4fe8-badb-d603063a19c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:34.737606 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c39273-ceca-4fe8-badb-d603063a19c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:34.739071 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c39273-ceca-4fe8-badb-d603063a19c5 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:34.758340 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-65c39273-ceca-4fe8-badb-d603063a19c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:34.758340 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:34.758340 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:34.758340 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:34.758340 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:34.758340 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:34.758340 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:34.758340 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:34.758340 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:34.758340 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:34.758340 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:34.758340 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:34.758340 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:34.758340 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:34.758340 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:34.764384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:34.764384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:34.764384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:34.764384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:34.764384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:34.764384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:34.764384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:34.764384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:34.764384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:34.764384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:34.764384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:34.764384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:34.764384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:34.764384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:34.764384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:34.764384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:34.764384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:34.765489 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:34.765489 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:34.765489 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:34.765489 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:34.765790 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c6fbc-6cf7-48aa-b7be-ce2782df607c None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:34.766177 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 402/811] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:34.766177 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c39273-ceca-4fe8-badb-d603063a19c5 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:34.796481 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55b5b6d-7569-4a03-9477-3e0fa82f5aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:34.797275 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55b5b6d-7569-4a03-9477-3e0fa82f5aff None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:34.797562 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55b5b6d-7569-4a03-9477-3e0fa82f5aff None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:34.797841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55b5b6d-7569-4a03-9477-3e0fa82f5aff None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:34.799300 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55b5b6d-7569-4a03-9477-3e0fa82f5aff None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:34.799645 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55b5b6d-7569-4a03-9477-3e0fa82f5aff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:34.801113 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55b5b6d-7569-4a03-9477-3e0fa82f5aff None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:34.822838 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 410/812] 10.222.0.13 () {56 vars in 1149 bytes} [Tue Jun 7 21:50:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:34.822838 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874456f2-1676-4369-8592-1ada4ebd45d7 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:34.822838 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874456f2-1676-4369-8592-1ada4ebd45d7 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:34.822838 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874456f2-1676-4369-8592-1ada4ebd45d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:34.836827 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-874456f2-1676-4369-8592-1ada4ebd45d7 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:34.838022 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-874456f2-1676-4369-8592-1ada4ebd45d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:34.859706 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:34.874927 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c39273-ceca-4fe8-badb-d603063a19c5 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:34.911551 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20e6964-82b0-4a97-8bba-ee679c7b137f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:34.912299 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e6964-82b0-4a97-8bba-ee679c7b137f None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:34.912414 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 411/813] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:50:34.913235 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874456f2-1676-4369-8592-1ada4ebd45d7 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:34.923756 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e6964-82b0-4a97-8bba-ee679c7b137f None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:34.924054 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e6964-82b0-4a97-8bba-ee679c7b137f None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:34.924794 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20e6964-82b0-4a97-8bba-ee679c7b137f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:34.925114 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20e6964-82b0-4a97-8bba-ee679c7b137f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:34.926501 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20e6964-82b0-4a97-8bba-ee679c7b137f None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:34.927932 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:34.936526 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 403/814] 10.222.0.13 () {56 vars in 1124 bytes} [Tue Jun 7 21:50:34 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:34.938720 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20e6964-82b0-4a97-8bba-ee679c7b137f None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:35.032588 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874456f2-1676-4369-8592-1ada4ebd45d7 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:35.044389 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20e6964-82b0-4a97-8bba-ee679c7b137f None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:35.071821 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8887fdc3-b845-4455-8d4b-724cfb2907c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:35.072503 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887fdc3-b845-4455-8d4b-724cfb2907c0 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:35.072789 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887fdc3-b845-4455-8d4b-724cfb2907c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:35.073067 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887fdc3-b845-4455-8d4b-724cfb2907c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:35.073795 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8887fdc3-b845-4455-8d4b-724cfb2907c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:35.074098 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8887fdc3-b845-4455-8d4b-724cfb2907c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:35.075491 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8887fdc3-b845-4455-8d4b-724cfb2907c0 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:35.080446 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff264f08-f245-40c1-89ed-8cef50648d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:35.081248 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff264f08-f245-40c1-89ed-8cef50648d93 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:35.081529 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff264f08-f245-40c1-89ed-8cef50648d93 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:35.081811 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff264f08-f245-40c1-89ed-8cef50648d93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:35.086968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 412/815] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:35.088304 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8887fdc3-b845-4455-8d4b-724cfb2907c0 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:35.094000 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff264f08-f245-40c1-89ed-8cef50648d93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:35.096008 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff264f08-f245-40c1-89ed-8cef50648d93 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:35.108942 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:35.114455 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-ff264f08-f245-40c1-89ed-8cef50648d93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:35.114455 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:35.114455 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:35.114455 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:35.114455 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:35.114455 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:35.114455 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:35.114455 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:35.114455 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:35.114455 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:35.114455 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:35.114455 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:35.114455 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:35.114455 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:35.114455 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:35.115517 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:35.115517 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:35.115517 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:35.115517 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:35.115517 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:35.115517 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:35.115517 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:35.115517 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:35.115517 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:35.115517 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:35.115517 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:35.115517 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:35.115517 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:35.115517 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:35.115517 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:35.115517 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:35.115517 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:35.116608 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:35.116608 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:35.116608 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:35.116608 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:35.120203 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 404/816] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:35.120884 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff264f08-f245-40c1-89ed-8cef50648d93 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:35.175306 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:35.205108 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8887fdc3-b845-4455-8d4b-724cfb2907c0 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:35.228773 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff264f08-f245-40c1-89ed-8cef50648d93 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:35.242594 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877e1db4-9679-4250-8c92-e2419592ac9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:35.243710 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e1db4-9679-4250-8c92-e2419592ac9f None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:35.243995 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e1db4-9679-4250-8c92-e2419592ac9f None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:35.244276 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e1db4-9679-4250-8c92-e2419592ac9f None admin] PATH_INFO: `/v3/services/network` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:35.244935 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877e1db4-9679-4250-8c92-e2419592ac9f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:35.246305 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877e1db4-9679-4250-8c92-e2419592ac9f None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:35.258485 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-877e1db4-9679-4250-8c92-e2419592ac9f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 21:50:35.258485 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:35.258485 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 21:50:35.258485 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 21:50:35.258485 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 21:50:35.258485 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 21:50:35.258485 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 21:50:35.258485 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:50:35.258485 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 21:50:35.258485 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:35.258485 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 21:50:35.258485 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:35.258485 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:35.258485 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:35.258485 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:35.258485 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:35.258485 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:35.259545 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:35.259545 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:35.259545 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:35.259545 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:35.259545 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:35.259545 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:35.259545 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 21:50:35.259545 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 21:50:35.259545 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 21:50:35.259545 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 21:50:35.259545 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:35.259545 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:35.259545 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:35.259545 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:35.259545 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:35.259545 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:35.260375 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:35.260375 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:35.260375 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:35.260375 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:35.260375 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:35.260375 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:35.260375 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:35.260375 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:35.260375 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:35.260375 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:35.260375 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 21:50:35.260375 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:50:35.260375 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 21:50:35.260375 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:35.260375 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 413/817] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:50:35 2022] GET /identity/v3/services/network => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:35.261170 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877e1db4-9679-4250-8c92-e2419592ac9f None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:35.273059 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b16ac80-4967-4f40-9a17-e3d99ac44855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:35.273942 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b16ac80-4967-4f40-9a17-e3d99ac44855 None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:35.274391 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b16ac80-4967-4f40-9a17-e3d99ac44855 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:35.274821 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b16ac80-4967-4f40-9a17-e3d99ac44855 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:35.275596 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b16ac80-4967-4f40-9a17-e3d99ac44855 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:35.276949 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b16ac80-4967-4f40-9a17-e3d99ac44855 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:35.311397 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 405/818] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Jun 7 21:50:35 2022] POST /identity/v3/endpoints => generated 346 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 21:50:35.311397 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b16ac80-4967-4f40-9a17-e3d99ac44855 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:35.364004 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:35.386591 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877e1db4-9679-4250-8c92-e2419592ac9f None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:35.433561 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407756b2-c781-4e4e-bada-2cf8858033c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:35.434289 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407756b2-c781-4e4e-bada-2cf8858033c9 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:35.434575 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407756b2-c781-4e4e-bada-2cf8858033c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:35.435722 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407756b2-c781-4e4e-bada-2cf8858033c9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:35.444348 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b16ac80-4967-4f40-9a17-e3d99ac44855 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:35.452751 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407756b2-c781-4e4e-bada-2cf8858033c9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:35.455051 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407756b2-c781-4e4e-bada-2cf8858033c9 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:35.467726 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:35.477169 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-407756b2-c781-4e4e-bada-2cf8858033c9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 21:50:35.477169 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:35.477169 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:35.477169 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:35.477169 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:35.477169 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:35.477169 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:35.477169 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:35.477169 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:35.477169 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:35.477169 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:35.477169 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:35.477169 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 21:50:35.477169 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 21:50:35.477169 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 21:50:35.477169 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 21:50:35.478378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:35.478378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:35.478378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 21:50:35.478378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:35.478378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 21:50:35.478378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:35.478378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:35.478378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:35.478378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:35.478378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:35.478378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:35.478378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:35.478378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:35.478378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:35.478378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:35.478378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:35.478378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:35.478378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:35.479651 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:35.479651 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:35.479651 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 21:50:35.479651 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 21:50:35.479651 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 21:50:35.479651 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 21:50:35.479651 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 21:50:35.479651 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 21:50:35.479651 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 21:50:35.479651 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:35.482868 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 414/819] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jun 7 21:50:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:35.484064 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407756b2-c781-4e4e-bada-2cf8858033c9 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:35.493849 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2bd76d-d17f-4ba8-9a1d-82adc81bbebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:35.494692 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2bd76d-d17f-4ba8-9a1d-82adc81bbebd None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:35.495095 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2bd76d-d17f-4ba8-9a1d-82adc81bbebd None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:35.495535 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2bd76d-d17f-4ba8-9a1d-82adc81bbebd None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:35.496408 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2bd76d-d17f-4ba8-9a1d-82adc81bbebd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:35.497071 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2bd76d-d17f-4ba8-9a1d-82adc81bbebd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:35.498627 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2bd76d-d17f-4ba8-9a1d-82adc81bbebd None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:35.519749 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 406/820] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:35.520942 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2bd76d-d17f-4ba8-9a1d-82adc81bbebd None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:35.630053 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407756b2-c781-4e4e-bada-2cf8858033c9 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:35.648733 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2bd76d-d17f-4ba8-9a1d-82adc81bbebd None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:35.676861 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44986414-507e-4c0b-942b-17cd48103aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:35.678154 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44986414-507e-4c0b-942b-17cd48103aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:35.678818 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44986414-507e-4c0b-942b-17cd48103aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:35.679301 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44986414-507e-4c0b-942b-17cd48103aa7 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:35.680784 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44986414-507e-4c0b-942b-17cd48103aa7 None admin] RBAC: Adding query filter params (name=network) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:35.681308 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44986414-507e-4c0b-942b-17cd48103aa7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:35.683685 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44986414-507e-4c0b-942b-17cd48103aa7 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:35.685987 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:35.688825 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c6309c-750c-46c5-a4c7-b20a9b912f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:35.690396 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6309c-750c-46c5-a4c7-b20a9b912f2d None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:35.690883 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6309c-750c-46c5-a4c7-b20a9b912f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:35.695358 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 415/821] 10.222.0.13 () {56 vars in 1112 bytes} [Tue Jun 7 21:50:35 2022] GET /identity/v3/services?name=network => generated 124 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:35.695358 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44986414-507e-4c0b-942b-17cd48103aa7 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:35.695983 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6309c-750c-46c5-a4c7-b20a9b912f2d None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:35.697442 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c6309c-750c-46c5-a4c7-b20a9b912f2d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:35.697992 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c6309c-750c-46c5-a4c7-b20a9b912f2d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:35.699674 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c6309c-750c-46c5-a4c7-b20a9b912f2d None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:35.722709 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 407/822] 10.222.0.13 () {56 vars in 1135 bytes} [Tue Jun 7 21:50:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:35.723966 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c6309c-750c-46c5-a4c7-b20a9b912f2d None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:35.762291 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:35.835484 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44986414-507e-4c0b-942b-17cd48103aa7 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:35.853921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c6309c-750c-46c5-a4c7-b20a9b912f2d None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:35.883124 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336dcf72-e12d-4d32-91f5-abf4858f5d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:35.884103 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336dcf72-e12d-4d32-91f5-abf4858f5d85 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:35.884389 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336dcf72-e12d-4d32-91f5-abf4858f5d85 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:35.884669 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336dcf72-e12d-4d32-91f5-abf4858f5d85 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:35.891396 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f034f204-9bc3-4e8e-8ad3-4403b35933b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:35.892911 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034f204-9bc3-4e8e-8ad3-4403b35933b3 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:35.893405 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034f204-9bc3-4e8e-8ad3-4403b35933b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:35.893679 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336dcf72-e12d-4d32-91f5-abf4858f5d85 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:35.894133 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034f204-9bc3-4e8e-8ad3-4403b35933b3 None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:35.895072 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f034f204-9bc3-4e8e-8ad3-4403b35933b3 None admin] RBAC: Adding query filter params (type=network) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:35.895180 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336dcf72-e12d-4d32-91f5-abf4858f5d85 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:35.895735 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f034f204-9bc3-4e8e-8ad3-4403b35933b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:35.897370 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f034f204-9bc3-4e8e-8ad3-4403b35933b3 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:35.907654 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 408/823] 10.222.0.13 () {56 vars in 1112 bytes} [Tue Jun 7 21:50:35 2022] GET /identity/v3/services?type=network => generated 350 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:35.909037 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f034f204-9bc3-4e8e-8ad3-4403b35933b3 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:35.911712 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-336dcf72-e12d-4d32-91f5-abf4858f5d85 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:35.911712 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:35.911712 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:35.911712 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:35.911712 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:35.911712 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:35.911712 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:35.911712 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:35.911712 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:35.911712 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:35.911712 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:35.911712 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:35.911712 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 21:50:35.911712 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 21:50:35.911712 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 21:50:35.912636 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 21:50:35.912636 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:35.912636 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:35.912636 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:35.912636 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:35.912636 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:35.912636 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:35.912636 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:35.912636 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:35.912636 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:35.912636 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:35.912636 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:35.912636 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:35.912636 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:35.912636 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:35.912636 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:35.912636 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:35.913926 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 21:50:35.913926 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 21:50:35.913926 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 21:50:35.913926 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 21:50:35.913926 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 21:50:35.913926 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 21:50:35.913926 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:35.913926 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:35.913926 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 416/824] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:50:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:35.913926 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336dcf72-e12d-4d32-91f5-abf4858f5d85 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:35.988031 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:36.010540 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:36.023470 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f034f204-9bc3-4e8e-8ad3-4403b35933b3 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:36.035679 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336dcf72-e12d-4d32-91f5-abf4858f5d85 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:36.063298 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68c372-7b67-4fa8-81e3-faf266dd89e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:36.064422 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68c372-7b67-4fa8-81e3-faf266dd89e5 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:36.064988 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68c372-7b67-4fa8-81e3-faf266dd89e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:36.065375 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68c372-7b67-4fa8-81e3-faf266dd89e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:36.074126 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62482bce-d2fd-4606-8b81-f8ae672ac30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:36.074990 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62482bce-d2fd-4606-8b81-f8ae672ac30c None admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:36.075478 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62482bce-d2fd-4606-8b81-f8ae672ac30c None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:36.075978 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62482bce-d2fd-4606-8b81-f8ae672ac30c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:36.076860 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62482bce-d2fd-4606-8b81-f8ae672ac30c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:36.078094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d68c372-7b67-4fa8-81e3-faf266dd89e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:36.078544 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62482bce-d2fd-4606-8b81-f8ae672ac30c None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:36.080427 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d68c372-7b67-4fa8-81e3-faf266dd89e5 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:36.103495 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 417/825] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Jun 7 21:50:35 2022] POST /identity/v3/endpoints => generated 327 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 21:50:36.104277 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-7d68c372-7b67-4fa8-81e3-faf266dd89e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:36.104277 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:36.104277 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:36.104277 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:36.104277 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:36.104277 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:36.104277 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:36.104277 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:36.104277 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:36.104277 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:36.104277 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:36.104277 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:36.104277 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:36.104277 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:36.104277 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:36.105165 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:36.105165 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:36.105165 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:36.105165 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:36.105165 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:36.105165 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:36.105165 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:36.105165 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:36.105165 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:36.105165 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:36.105165 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:36.105165 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:36.105165 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:36.105165 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:36.105165 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:36.105165 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:36.105165 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:36.106076 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:36.106076 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:36.106076 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:36.106076 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:36.106820 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62482bce-d2fd-4606-8b81-f8ae672ac30c None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:36.115423 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 409/826] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:36.118652 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68c372-7b67-4fa8-81e3-faf266dd89e5 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:36.234701 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62482bce-d2fd-4606-8b81-f8ae672ac30c None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:36.251174 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:36.278318 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1605b26-1b0a-46f8-87ce-0496d4a02546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:36.279195 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1605b26-1b0a-46f8-87ce-0496d4a02546 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:36.279614 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1605b26-1b0a-46f8-87ce-0496d4a02546 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:36.279997 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1605b26-1b0a-46f8-87ce-0496d4a02546 None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:36.281068 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1605b26-1b0a-46f8-87ce-0496d4a02546 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:36.281488 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1605b26-1b0a-46f8-87ce-0496d4a02546 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:36.283607 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1605b26-1b0a-46f8-87ce-0496d4a02546 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:36.297783 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 418/827] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jun 7 21:50:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:36.304019 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1605b26-1b0a-46f8-87ce-0496d4a02546 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:36.306393 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68c372-7b67-4fa8-81e3-faf266dd89e5 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:36.374195 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d0925-42f9-4edf-869b-43d1df421364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:36.375956 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d0925-42f9-4edf-869b-43d1df421364 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:36.376247 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d0925-42f9-4edf-869b-43d1df421364 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:36.376596 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d0925-42f9-4edf-869b-43d1df421364 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:36.379148 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073d0925-42f9-4edf-869b-43d1df421364 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:36.379492 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073d0925-42f9-4edf-869b-43d1df421364 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:36.382019 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073d0925-42f9-4edf-869b-43d1df421364 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:36.403701 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:36.414466 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 410/828] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:36.419635 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d0925-42f9-4edf-869b-43d1df421364 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:36.499006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1605b26-1b0a-46f8-87ce-0496d4a02546 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:36.544517 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6a9312-10a9-4e85-b337-ca89645c8528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:36.545810 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a9312-10a9-4e85-b337-ca89645c8528 None admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:36.546327 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a9312-10a9-4e85-b337-ca89645c8528 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:36.547841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a9312-10a9-4e85-b337-ca89645c8528 None admin] PATH_INFO: `/v3/projects/0219dbda54274fdc9bdccbd960c30c6b/users/86619a80df8841a3ba7d8471ac38056a/roles/2c7f73c184d0406bbf2899465953342b` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:36.565984 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d0925-42f9-4edf-869b-43d1df421364 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:36.566588 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6a9312-10a9-4e85-b337-ca89645c8528 None admin] RBAC: Authorizing `identity:create_grant(project_id=0219dbda54274fdc9bdccbd960c30c6b, user_id=86619a80df8841a3ba7d8471ac38056a, role_id=2c7f73c184d0406bbf2899465953342b)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:36.568559 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6a9312-10a9-4e85-b337-ca89645c8528 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:36.587473 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 419/829] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jun 7 21:50:36 2022] PUT /identity/v3/projects/0219dbda54274fdc9bdccbd960c30c6b/users/86619a80df8841a3ba7d8471ac38056a/roles/2c7f73c184d0406bbf2899465953342b => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 21:50:36.618695 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd95371-08b1-4857-8c8d-ba67681c23ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:36.619564 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd95371-08b1-4857-8c8d-ba67681c23ce None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:36.619850 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd95371-08b1-4857-8c8d-ba67681c23ce None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:36.620130 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd95371-08b1-4857-8c8d-ba67681c23ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:36.629934 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd95371-08b1-4857-8c8d-ba67681c23ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:36.631486 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd95371-08b1-4857-8c8d-ba67681c23ce None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:36.655290 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-0cd95371-08b1-4857-8c8d-ba67681c23ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:36.655290 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:36.655290 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:36.655290 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:36.655290 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:36.655290 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:36.655290 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:36.655290 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:36.655290 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:36.655290 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:36.655290 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:36.655290 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:36.655290 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 21:50:36.655290 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 21:50:36.655290 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 21:50:36.656384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 21:50:36.656384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:36.656384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:36.656384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:36.656384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:36.656384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:36.656384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:36.656384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:36.656384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:36.656384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:36.656384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:36.656384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:36.656384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:36.656384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:36.656384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:36.656384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:36.656384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:36.657284 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 21:50:36.657284 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 21:50:36.657284 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 21:50:36.657284 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 21:50:36.657284 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 21:50:36.657284 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 21:50:36.657284 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:36.657284 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:36.657284 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 411/830] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:50:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:36.665949 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6a9312-10a9-4e85-b337-ca89645c8528 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:36.702763 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:36.806204 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6a9312-10a9-4e85-b337-ca89645c8528 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:36.827644 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efab56e-54c6-4c37-9979-6e173ec48276 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:36.828604 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efab56e-54c6-4c37-9979-6e173ec48276 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:36.828890 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efab56e-54c6-4c37-9979-6e173ec48276 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:36.830967 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 412/831] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:36.837970 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f172e3fb-163c-4631-bd60-b895e4b2ffac None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:36.838259 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f172e3fb-163c-4631-bd60-b895e4b2ffac None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:36.838542 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f172e3fb-163c-4631-bd60-b895e4b2ffac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:36.845137 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cdfa85-2c39-4b9e-8810-c3a2abccab2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:36.847260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cdfa85-2c39-4b9e-8810-c3a2abccab2f None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:36.847260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cdfa85-2c39-4b9e-8810-c3a2abccab2f None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:36.847512 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cdfa85-2c39-4b9e-8810-c3a2abccab2f None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:36.848259 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cdfa85-2c39-4b9e-8810-c3a2abccab2f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:36.848570 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cdfa85-2c39-4b9e-8810-c3a2abccab2f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:36.850015 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cdfa85-2c39-4b9e-8810-c3a2abccab2f None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:36.863574 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-f172e3fb-163c-4631-bd60-b895e4b2ffac None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:36.864651 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 420/832] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jun 7 21:50:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:36.864937 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-f172e3fb-163c-4631-bd60-b895e4b2ffac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:36.879350 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cdfa85-2c39-4b9e-8810-c3a2abccab2f None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:36.940002 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 413/833] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:36 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:50:36.940582 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:36.963038 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f172e3fb-163c-4631-bd60-b895e4b2ffac None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:36.977698 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:37.001006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cdfa85-2c39-4b9e-8810-c3a2abccab2f None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:37.038563 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae674d3-f9dc-4984-9337-2fea7771b553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:37.039939 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae674d3-f9dc-4984-9337-2fea7771b553 None admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:37.040428 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae674d3-f9dc-4984-9337-2fea7771b553 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:37.040925 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae674d3-f9dc-4984-9337-2fea7771b553 None admin] PATH_INFO: `/v3/projects/0219dbda54274fdc9bdccbd960c30c6b/users/7a1261563b6d4e50a4ffeea6a32bdb01/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:37.043417 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae674d3-f9dc-4984-9337-2fea7771b553 None admin] RBAC: Authorizing `identity:create_grant(project_id=0219dbda54274fdc9bdccbd960c30c6b, user_id=7a1261563b6d4e50a4ffeea6a32bdb01, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:37.045142 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae674d3-f9dc-4984-9337-2fea7771b553 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:37.061609 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 421/834] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jun 7 21:50:36 2022] PUT /identity/v3/projects/0219dbda54274fdc9bdccbd960c30c6b/users/7a1261563b6d4e50a4ffeea6a32bdb01/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 21:50:37.088385 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f172e3fb-163c-4631-bd60-b895e4b2ffac None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:37.130943 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c41ed25-c410-4a35-b962-d4989befee07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:37.132252 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c41ed25-c410-4a35-b962-d4989befee07 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:37.132788 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c41ed25-c410-4a35-b962-d4989befee07 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:37.133300 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c41ed25-c410-4a35-b962-d4989befee07 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:37.134256 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c41ed25-c410-4a35-b962-d4989befee07 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:37.136008 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c41ed25-c410-4a35-b962-d4989befee07 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:37.150463 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-9c41ed25-c410-4a35-b962-d4989befee07 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 21:50:37.150463 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:37.150463 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 21:50:37.150463 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 21:50:37.150463 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 21:50:37.150463 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 21:50:37.150463 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 21:50:37.150463 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:50:37.150463 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 21:50:37.150463 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:37.150463 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 21:50:37.150463 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:37.150463 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:37.150463 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:37.150463 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:37.150463 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:37.151557 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:37.151557 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:37.151557 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:37.151557 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:37.151557 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:37.151557 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:37.151557 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:37.151557 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 21:50:37.151557 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 21:50:37.151557 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 21:50:37.151557 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 21:50:37.151557 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:37.151557 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:37.151557 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:37.151557 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:37.151557 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:37.152421 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:37.152421 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:37.152421 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:37.152421 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:37.152421 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:37.152421 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:37.152421 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:37.152421 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:37.152421 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:37.152421 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:37.152421 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:37.152421 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 21:50:37.152421 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:50:37.152421 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 21:50:37.152421 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:37.155550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 414/835] 10.222.0.13 () {56 vars in 1132 bytes} [Tue Jun 7 21:50:36 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:50:37.163065 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae674d3-f9dc-4984-9337-2fea7771b553 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:37.282582 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae674d3-f9dc-4984-9337-2fea7771b553 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:37.322975 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a7072b-1d48-4b84-b4e2-857159875624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:37.323993 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7072b-1d48-4b84-b4e2-857159875624 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:37.324419 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7072b-1d48-4b84-b4e2-857159875624 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:37.324838 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7072b-1d48-4b84-b4e2-857159875624 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:37.325708 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a7072b-1d48-4b84-b4e2-857159875624 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:37.326155 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a7072b-1d48-4b84-b4e2-857159875624 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:37.327675 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a7072b-1d48-4b84-b4e2-857159875624 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:37.333456 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 422/836] 10.222.0.13 () {56 vars in 1126 bytes} [Tue Jun 7 21:50:37 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:37.341635 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c41ed25-c410-4a35-b962-d4989befee07 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:37.404279 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:37.451880 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c41ed25-c410-4a35-b962-d4989befee07 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:37.489959 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5306c8-a1c6-4c61-86c6-f74b112bc45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:37.490633 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5306c8-a1c6-4c61-86c6-f74b112bc45f None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:37.490920 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5306c8-a1c6-4c61-86c6-f74b112bc45f None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:37.491203 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5306c8-a1c6-4c61-86c6-f74b112bc45f None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:37.491951 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5306c8-a1c6-4c61-86c6-f74b112bc45f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:37.492262 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5306c8-a1c6-4c61-86c6-f74b112bc45f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:37.493643 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5306c8-a1c6-4c61-86c6-f74b112bc45f None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:37.498843 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 415/837] 10.222.0.13 () {56 vars in 1126 bytes} [Tue Jun 7 21:50:37 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:37.723475 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d2762-a21d-4a82-9e40-6096c7e97d2a None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:37.723676 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d2762-a21d-4a82-9e40-6096c7e97d2a None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:37.723952 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d2762-a21d-4a82-9e40-6096c7e97d2a None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:37.725585 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 423/838] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:37.730845 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159451f2-8e91-4564-a976-be6700dff29e None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:37.731135 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159451f2-8e91-4564-a976-be6700dff29e None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:37.731450 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159451f2-8e91-4564-a976-be6700dff29e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:37.749371 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-159451f2-8e91-4564-a976-be6700dff29e None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:37.750542 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-159451f2-8e91-4564-a976-be6700dff29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:37.819313 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 416/839] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:37 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:50:37.841493 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9d2762-a21d-4a82-9e40-6096c7e97d2a None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:37.852149 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:37.958998 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9d2762-a21d-4a82-9e40-6096c7e97d2a None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:37.996310 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f02fa0-9286-42ed-9451-018f4f36f574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:37.997039 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f02fa0-9286-42ed-9451-018f4f36f574 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:37.997321 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f02fa0-9286-42ed-9451-018f4f36f574 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:37.997600 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f02fa0-9286-42ed-9451-018f4f36f574 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:37.998254 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f02fa0-9286-42ed-9451-018f4f36f574 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:37.999661 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f02fa0-9286-42ed-9451-018f4f36f574 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:38.012925 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-25f02fa0-9286-42ed-9451-018f4f36f574 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 21:50:38.012925 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:38.012925 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 21:50:38.012925 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 21:50:38.012925 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 21:50:38.012925 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 21:50:38.012925 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 21:50:38.012925 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:50:38.012925 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 21:50:38.012925 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:38.012925 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 21:50:38.012925 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:38.012925 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:38.012925 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:38.012925 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:38.012925 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:38.012925 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:38.013975 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:38.013975 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:38.013975 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:38.013975 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:38.013975 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:38.013975 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:38.013975 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 21:50:38.013975 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 21:50:38.013975 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 21:50:38.013975 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 21:50:38.013975 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:38.013975 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:38.013975 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:38.013975 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:38.013975 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:38.013975 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:38.014824 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:38.014824 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:38.014824 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:38.014824 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:38.014824 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:38.014824 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:38.014824 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:38.014824 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:38.014824 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:38.014824 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:38.014824 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 21:50:38.014824 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:50:38.014824 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 21:50:38.014824 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:38.014824 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 424/840] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Jun 7 21:50:37 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:38.020783 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159451f2-8e91-4564-a976-be6700dff29e None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:38.072786 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:38.127939 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159451f2-8e91-4564-a976-be6700dff29e None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:38.163108 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a5a5b1-7aa6-4b19-ae66-92ad28bb8a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:38.164085 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a5a5b1-7aa6-4b19-ae66-92ad28bb8a10 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:38.164629 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a5a5b1-7aa6-4b19-ae66-92ad28bb8a10 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:38.165098 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a5a5b1-7aa6-4b19-ae66-92ad28bb8a10 None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:38.166026 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a5a5b1-7aa6-4b19-ae66-92ad28bb8a10 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:38.166532 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a5a5b1-7aa6-4b19-ae66-92ad28bb8a10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:38.168095 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a5a5b1-7aa6-4b19-ae66-92ad28bb8a10 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:38.173110 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 417/841] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Jun 7 21:50:38 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:38.180175 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f02fa0-9286-42ed-9451-018f4f36f574 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:38.252193 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:38.291221 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f02fa0-9286-42ed-9451-018f4f36f574 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:38.326518 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8189aad-0711-48a4-ada2-97f3cc387493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:38.327202 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8189aad-0711-48a4-ada2-97f3cc387493 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:38.327517 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8189aad-0711-48a4-ada2-97f3cc387493 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:38.327792 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8189aad-0711-48a4-ada2-97f3cc387493 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:38.328517 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8189aad-0711-48a4-ada2-97f3cc387493 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:38.328822 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8189aad-0711-48a4-ada2-97f3cc387493 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:38.331016 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8189aad-0711-48a4-ada2-97f3cc387493 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:38.336636 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 425/842] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Jun 7 21:50:38 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:39.051503 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf80d3f-55dc-491a-ba44-3a33ceb7f868 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:39.052076 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf80d3f-55dc-491a-ba44-3a33ceb7f868 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:39.052076 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf80d3f-55dc-491a-ba44-3a33ceb7f868 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:39.053709 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 418/843] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:39.059120 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f94a7b-f4fb-4a36-9372-f59cfcd29479 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:39.059595 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f94a7b-f4fb-4a36-9372-f59cfcd29479 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:39.059772 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f94a7b-f4fb-4a36-9372-f59cfcd29479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:39.077817 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-22f94a7b-f4fb-4a36-9372-f59cfcd29479 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:39.079031 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-22f94a7b-f4fb-4a36-9372-f59cfcd29479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:39.155556 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 426/844] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:39 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:50:39.176833 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf80d3f-55dc-491a-ba44-3a33ceb7f868 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:39.286961 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf80d3f-55dc-491a-ba44-3a33ceb7f868 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:39.300879 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:39.326995 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ec57f6-c40e-49b9-aea0-0cbe606013b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:39.327753 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ec57f6-c40e-49b9-aea0-0cbe606013b3 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:39.328041 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ec57f6-c40e-49b9-aea0-0cbe606013b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:39.328323 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ec57f6-c40e-49b9-aea0-0cbe606013b3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:39.339750 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ec57f6-c40e-49b9-aea0-0cbe606013b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:50:39.348034 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-d3ec57f6-c40e-49b9-aea0-0cbe606013b3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 21:50:39.348034 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:39.348034 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:39.348034 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:39.348034 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:39.348034 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:39.348034 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:39.348034 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:39.348034 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:39.348034 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:39.348034 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:39.348034 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:39.348034 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:50:39.348034 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:50:39.348034 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:50:39.348034 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:50:39.349052 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:50:39.349052 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:50:39.349052 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:50:39.349052 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:50:39.349052 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:39.349052 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:39.349052 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:39.349052 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:39.349052 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:39.349052 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:39.349052 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:39.349052 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:39.349052 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:39.349052 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:39.349052 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:39.349052 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:39.349969 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:39.349969 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:39.349969 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:39.349969 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:39.349969 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:50:39.349969 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:50:39.349969 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:50:39.349969 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:50:39.349969 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:50:39.349969 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:50:39.349969 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 21:50:39.349969 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:39.349969 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 419/845] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jun 7 21:50:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:39.355172 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f94a7b-f4fb-4a36-9372-f59cfcd29479 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:39.433994 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:39.473831 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f94a7b-f4fb-4a36-9372-f59cfcd29479 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:39.509791 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e8a11-0a6b-4c6d-bd63-507113c0a7fe None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:39.510091 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e8a11-0a6b-4c6d-bd63-507113c0a7fe None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:39.510381 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e8a11-0a6b-4c6d-bd63-507113c0a7fe None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:39.510979 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88b92de-9981-4d48-8727-fb6de0da1d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:39.511839 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b92de-9981-4d48-8727-fb6de0da1d6e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:39.512060 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 420/846] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:39.512421 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b92de-9981-4d48-8727-fb6de0da1d6e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:39.512853 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b92de-9981-4d48-8727-fb6de0da1d6e None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:39.513854 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88b92de-9981-4d48-8727-fb6de0da1d6e None admin] RBAC: Adding query filter params (name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:39.514351 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88b92de-9981-4d48-8727-fb6de0da1d6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:39.515975 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88b92de-9981-4d48-8727-fb6de0da1d6e None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:39.517169 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470bafe-8e77-4412-b9df-278cacf32184 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:39.517978 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470bafe-8e77-4412-b9df-278cacf32184 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:39.518267 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470bafe-8e77-4412-b9df-278cacf32184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:39.527727 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 427/847] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Jun 7 21:50:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:39.533850 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88b92de-9981-4d48-8727-fb6de0da1d6e None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:39.536530 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-5470bafe-8e77-4412-b9df-278cacf32184 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:39.537762 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-5470bafe-8e77-4412-b9df-278cacf32184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:39.619856 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 421/848] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:39 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 21:50:39.636989 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5470bafe-8e77-4412-b9df-278cacf32184 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:39.649901 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88b92de-9981-4d48-8727-fb6de0da1d6e None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:39.679657 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:39.688279 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc4c044-d2b9-415a-b4e5-26247e72389b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:39.689458 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4c044-d2b9-415a-b4e5-26247e72389b None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:39.689905 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4c044-d2b9-415a-b4e5-26247e72389b None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:39.690186 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4c044-d2b9-415a-b4e5-26247e72389b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:39.702829 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc4c044-d2b9-415a-b4e5-26247e72389b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:39.704307 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc4c044-d2b9-415a-b4e5-26247e72389b None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:39.724422 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-fcc4c044-d2b9-415a-b4e5-26247e72389b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:39.724422 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:39.724422 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:39.724422 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:39.724422 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:39.724422 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:39.724422 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:39.724422 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:39.724422 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:39.724422 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:39.724422 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:39.724422 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:39.724422 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:39.724422 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:39.724422 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:39.725379 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:39.725379 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:39.725379 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:39.725379 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:39.725379 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:39.725379 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:39.725379 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:39.725379 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:39.725379 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:39.725379 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:39.725379 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:39.725379 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:39.725379 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:39.725379 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:39.725379 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:39.725379 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:39.725379 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:39.726782 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:39.726782 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:39.726782 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:39.726782 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:39.726782 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 428/849] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:39.733639 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc4c044-d2b9-415a-b4e5-26247e72389b None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:39.746490 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5470bafe-8e77-4412-b9df-278cacf32184 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:39.749441 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:39.802763 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a6db95-e1f1-48bd-b9f7-90c3d9ea4964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:39.804045 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6db95-e1f1-48bd-b9f7-90c3d9ea4964 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:39.804344 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6db95-e1f1-48bd-b9f7-90c3d9ea4964 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:39.804629 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6db95-e1f1-48bd-b9f7-90c3d9ea4964 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:39.819927 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a6db95-e1f1-48bd-b9f7-90c3d9ea4964 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:50:39.830593 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-e2a6db95-e1f1-48bd-b9f7-90c3d9ea4964 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:50:39.830593 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:39.830593 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:39.830593 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:39.830593 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:39.830593 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:39.830593 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:39.830593 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:39.830593 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:39.830593 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:39.830593 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:39.830593 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:39.830593 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:50:39.830593 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:50:39.830593 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:50:39.830593 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:50:39.831609 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:50:39.831609 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:50:39.831609 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:50:39.831609 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:50:39.831609 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:39.831609 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:39.831609 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:39.831609 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:39.831609 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:39.831609 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:39.831609 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:39.831609 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:39.831609 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:39.831609 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:39.831609 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:39.831609 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:39.832470 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:39.832470 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:39.832470 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:39.832470 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:39.832470 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:50:39.832470 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:50:39.832470 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:50:39.832470 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:50:39.832470 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:50:39.832470 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:50:39.832470 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:50:39.832470 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:39.833475 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 422/850] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jun 7 21:50:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:50:39.840986 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a6db95-e1f1-48bd-b9f7-90c3d9ea4964 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:39.866566 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc4c044-d2b9-415a-b4e5-26247e72389b None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:39.905146 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24613a9c-4883-4e5c-8865-3a1e7003ea44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:39.905786 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24613a9c-4883-4e5c-8865-3a1e7003ea44 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:39.906008 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24613a9c-4883-4e5c-8865-3a1e7003ea44 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:39.906287 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24613a9c-4883-4e5c-8865-3a1e7003ea44 None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:39.907009 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24613a9c-4883-4e5c-8865-3a1e7003ea44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:39.907313 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24613a9c-4883-4e5c-8865-3a1e7003ea44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:39.909111 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24613a9c-4883-4e5c-8865-3a1e7003ea44 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:39.922379 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 429/851] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:39.929128 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36deaad2-fab1-4547-96cb-6848806222fa None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:39.929417 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36deaad2-fab1-4547-96cb-6848806222fa None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:39.929695 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36deaad2-fab1-4547-96cb-6848806222fa None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:39.931147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 430/852] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:39.932167 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36deaad2-fab1-4547-96cb-6848806222fa None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:39.968928 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a6db95-e1f1-48bd-b9f7-90c3d9ea4964 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:39.979320 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:40.006330 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d2228-b1cd-428c-b6ad-22aad15c89f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:40.007001 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d2228-b1cd-428c-b6ad-22aad15c89f9 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:40.007281 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d2228-b1cd-428c-b6ad-22aad15c89f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:40.007583 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d2228-b1cd-428c-b6ad-22aad15c89f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:40.008502 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669d2228-b1cd-428c-b6ad-22aad15c89f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:40.008812 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669d2228-b1cd-428c-b6ad-22aad15c89f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:40.010303 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669d2228-b1cd-428c-b6ad-22aad15c89f9 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:40.024698 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 423/853] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:50:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:50:40.024698 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6844ad-17f8-4076-a2cc-b2b459eb3385 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:40.024698 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6844ad-17f8-4076-a2cc-b2b459eb3385 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:40.024698 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6844ad-17f8-4076-a2cc-b2b459eb3385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:40.041663 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-7b6844ad-17f8-4076-a2cc-b2b459eb3385 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:40.043005 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-7b6844ad-17f8-4076-a2cc-b2b459eb3385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:40.048499 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36deaad2-fab1-4547-96cb-6848806222fa None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:40.061877 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:40.085816 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197d7be9-c604-4fdc-94e4-1ba3d679ad4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:40.086730 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d7be9-c604-4fdc-94e4-1ba3d679ad4b None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:40.087087 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d7be9-c604-4fdc-94e4-1ba3d679ad4b None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:40.087561 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d7be9-c604-4fdc-94e4-1ba3d679ad4b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:40.095673 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197d7be9-c604-4fdc-94e4-1ba3d679ad4b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:40.097612 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197d7be9-c604-4fdc-94e4-1ba3d679ad4b None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:40.113824 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-197d7be9-c604-4fdc-94e4-1ba3d679ad4b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 21:50:40.113824 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:40.113824 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:40.113824 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:40.113824 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:40.113824 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:40.113824 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:40.113824 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:40.113824 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:40.113824 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:40.113824 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:40.113824 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:40.113824 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 21:50:40.113824 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 21:50:40.113824 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 21:50:40.113824 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 21:50:40.114860 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:40.114860 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:40.114860 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 21:50:40.114860 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:40.114860 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 21:50:40.114860 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:40.114860 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:40.114860 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:40.114860 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:40.114860 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:40.114860 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:40.114860 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:40.114860 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:40.114860 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:40.114860 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:40.114860 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:40.114860 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:40.114860 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:40.115851 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:40.115851 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:40.115851 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 21:50:40.115851 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 21:50:40.115851 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 21:50:40.115851 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 21:50:40.115851 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 21:50:40.115851 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 21:50:40.115851 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 21:50:40.115851 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:40.115851 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 431/854] 10.222.0.13 () {56 vars in 1117 bytes} [Tue Jun 7 21:50:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:40.115851 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197d7be9-c604-4fdc-94e4-1ba3d679ad4b None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:40.122027 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 424/855] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:40 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:50:40.123100 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6844ad-17f8-4076-a2cc-b2b459eb3385 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:40.232073 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6844ad-17f8-4076-a2cc-b2b459eb3385 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:40.234088 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:40.242343 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197d7be9-c604-4fdc-94e4-1ba3d679ad4b None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:40.270079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a2aba0-c752-4345-82f5-e516512ed203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:40.270787 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2aba0-c752-4345-82f5-e516512ed203 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:40.271067 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2aba0-c752-4345-82f5-e516512ed203 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:40.271379 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2aba0-c752-4345-82f5-e516512ed203 None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:40.272234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a2aba0-c752-4345-82f5-e516512ed203 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:40.272550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a2aba0-c752-4345-82f5-e516512ed203 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:40.274031 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a2aba0-c752-4345-82f5-e516512ed203 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:40.281883 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ed228f-56d5-4236-b2ef-19790307c21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:40.283244 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed228f-56d5-4236-b2ef-19790307c21f None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:40.283599 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed228f-56d5-4236-b2ef-19790307c21f None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:40.283918 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed228f-56d5-4236-b2ef-19790307c21f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:40.295251 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ed228f-56d5-4236-b2ef-19790307c21f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:40.296015 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 425/856] 10.222.0.13 () {56 vars in 1149 bytes} [Tue Jun 7 21:50:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:40.296764 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ed228f-56d5-4236-b2ef-19790307c21f None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:40.297311 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a2aba0-c752-4345-82f5-e516512ed203 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:40.318799 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:40.319417 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-07ed228f-56d5-4236-b2ef-19790307c21f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:40.319417 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:40.319417 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:40.319417 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:40.319417 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:40.319417 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:40.319417 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:40.319417 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:40.319417 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:40.319417 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:40.319417 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:40.319417 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:40.319417 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:40.319417 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:40.319417 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:40.320297 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:40.320297 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:40.320297 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:40.320297 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:40.320297 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:40.320297 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:40.320297 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:40.320297 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:40.320297 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:40.320297 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:40.320297 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:40.320297 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:40.320297 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:40.320297 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:40.320297 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:40.320297 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:40.320297 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:40.321244 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:40.321244 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:40.321244 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:40.321244 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:40.325389 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 432/857] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:40.326004 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ed228f-56d5-4236-b2ef-19790307c21f None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:40.418695 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a2aba0-c752-4345-82f5-e516512ed203 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:40.432222 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ed228f-56d5-4236-b2ef-19790307c21f None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:40.454080 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c0ff59-9eb1-4594-9612-c51f957f6003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:40.455352 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0ff59-9eb1-4594-9612-c51f957f6003 None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:40.455645 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0ff59-9eb1-4594-9612-c51f957f6003 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:40.456129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0ff59-9eb1-4594-9612-c51f957f6003 None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:40.456855 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c0ff59-9eb1-4594-9612-c51f957f6003 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:40.458213 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c0ff59-9eb1-4594-9612-c51f957f6003 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:40.467779 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f13c82f-fe54-43d1-801c-7d9789b9b495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:40.468437 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f13c82f-fe54-43d1-801c-7d9789b9b495 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:40.468717 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f13c82f-fe54-43d1-801c-7d9789b9b495 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:40.468998 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f13c82f-fe54-43d1-801c-7d9789b9b495 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:40.474842 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 426/858] 10.222.0.13 () {60 vars in 1141 bytes} [Tue Jun 7 21:50:40 2022] POST /identity/v3/services => generated 269 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 21:50:40.475902 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c0ff59-9eb1-4594-9612-c51f957f6003 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:40.496494 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:40.568405 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f13c82f-fe54-43d1-801c-7d9789b9b495 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:40.570495 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f13c82f-fe54-43d1-801c-7d9789b9b495 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:40.572247 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:40.625295 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c0ff59-9eb1-4594-9612-c51f957f6003 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:40.667866 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a7792-84d9-4967-b270-ea4810e55066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:40.668544 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a7792-84d9-4967-b270-ea4810e55066 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:40.668830 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a7792-84d9-4967-b270-ea4810e55066 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:40.669105 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a7792-84d9-4967-b270-ea4810e55066 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:40.669836 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223a7792-84d9-4967-b270-ea4810e55066 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:40.670143 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223a7792-84d9-4967-b270-ea4810e55066 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:40.671563 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223a7792-84d9-4967-b270-ea4810e55066 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:40.687101 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 427/859] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:40.688298 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e58b6-246e-45db-a67f-4044886906dd None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:40.688589 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e58b6-246e-45db-a67f-4044886906dd None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:40.688929 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e58b6-246e-45db-a67f-4044886906dd None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:40.691084 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 428/860] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:40.694164 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853e58b6-246e-45db-a67f-4044886906dd None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:40.695795 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 433/861] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:50:40 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 21:50:40.700101 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39627ba-acb8-4cf3-bbb3-9fbc620af130 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:40.700101 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39627ba-acb8-4cf3-bbb3-9fbc620af130 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:40.700101 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39627ba-acb8-4cf3-bbb3-9fbc620af130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:40.714862 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-c39627ba-acb8-4cf3-bbb3-9fbc620af130 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:40.716123 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c39627ba-acb8-4cf3-bbb3-9fbc620af130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:40.777759 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:40.810658 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 434/862] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:40 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:50:40.810658 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39627ba-acb8-4cf3-bbb3-9fbc620af130 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:40.820550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853e58b6-246e-45db-a67f-4044886906dd None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:40.853033 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:40.863548 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2802c-3d85-4efa-a00c-b6411db0d57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:40.864392 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2802c-3d85-4efa-a00c-b6411db0d57e None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:40.864682 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2802c-3d85-4efa-a00c-b6411db0d57e None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:40.864961 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2802c-3d85-4efa-a00c-b6411db0d57e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:40.877037 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea2802c-3d85-4efa-a00c-b6411db0d57e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:40.879040 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea2802c-3d85-4efa-a00c-b6411db0d57e None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:40.895316 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-1ea2802c-3d85-4efa-a00c-b6411db0d57e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 21:50:40.895316 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:40.895316 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:40.895316 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:40.895316 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:40.895316 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:40.895316 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:40.895316 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:40.895316 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:40.895316 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:40.895316 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:40.895316 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:40.895316 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 21:50:40.895316 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 21:50:40.895316 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 21:50:40.895316 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 21:50:40.896318 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:40.896318 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:40.896318 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 21:50:40.896318 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:40.896318 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 21:50:40.896318 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:40.896318 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:40.896318 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:40.896318 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:40.896318 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:40.896318 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:40.896318 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:40.896318 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:40.896318 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:40.896318 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:40.896318 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:40.896318 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:40.896318 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:40.897282 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:40.897282 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:40.897282 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 21:50:40.897282 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 21:50:40.897282 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 21:50:40.897282 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 21:50:40.897282 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 21:50:40.897282 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 21:50:40.897282 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 21:50:40.897282 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:40.903176 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 429/863] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jun 7 21:50:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:40.903176 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2802c-3d85-4efa-a00c-b6411db0d57e None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:40.914284 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39627ba-acb8-4cf3-bbb3-9fbc620af130 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:40.951760 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2e19b1-2e8e-429f-a5a5-a301a11d9250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:40.952531 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e19b1-2e8e-429f-a5a5-a301a11d9250 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:40.952887 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e19b1-2e8e-429f-a5a5-a301a11d9250 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:40.953125 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e19b1-2e8e-429f-a5a5-a301a11d9250 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:40.963616 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2e19b1-2e8e-429f-a5a5-a301a11d9250 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:40.965070 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2e19b1-2e8e-429f-a5a5-a301a11d9250 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:40.984097 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-5b2e19b1-2e8e-429f-a5a5-a301a11d9250 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:40.984097 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:40.984097 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:40.984097 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:40.984097 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:40.984097 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:40.984097 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:40.984097 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:40.984097 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:40.984097 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:40.984097 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:40.984097 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:40.984097 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:40.984097 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:40.984097 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:40.985027 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:40.985027 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:40.985027 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:40.985027 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:40.985027 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:40.985027 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:40.985027 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:40.985027 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:40.985027 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:40.985027 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:40.985027 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:40.985027 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:40.985027 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:40.985027 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:40.985027 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:40.985027 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:40.985027 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:40.986818 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:40.986818 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:40.986818 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:40.986818 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:40.986818 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 435/864] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:40.986818 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2e19b1-2e8e-429f-a5a5-a301a11d9250 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:41.014901 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2802c-3d85-4efa-a00c-b6411db0d57e None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:41.021253 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:41.052512 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfb5d18-d04e-4a7c-8006-091d15f3fa0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:41.053179 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb5d18-d04e-4a7c-8006-091d15f3fa0a None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:41.053465 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb5d18-d04e-4a7c-8006-091d15f3fa0a None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:41.053741 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb5d18-d04e-4a7c-8006-091d15f3fa0a None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:41.054585 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfb5d18-d04e-4a7c-8006-091d15f3fa0a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:41.056095 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfb5d18-d04e-4a7c-8006-091d15f3fa0a None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:41.080166 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 430/865] 10.222.0.13 () {60 vars in 1141 bytes} [Tue Jun 7 21:50:40 2022] POST /identity/v3/services => generated 251 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 21:50:41.080166 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfb5d18-d04e-4a7c-8006-091d15f3fa0a None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:41.098455 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2e19b1-2e8e-429f-a5a5-a301a11d9250 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:41.144347 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928adc8c-4f4a-4cf1-9c6e-09227521ba3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:41.145082 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928adc8c-4f4a-4cf1-9c6e-09227521ba3b None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:41.145361 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928adc8c-4f4a-4cf1-9c6e-09227521ba3b None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:41.145677 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928adc8c-4f4a-4cf1-9c6e-09227521ba3b None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:41.146528 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928adc8c-4f4a-4cf1-9c6e-09227521ba3b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:41.146835 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928adc8c-4f4a-4cf1-9c6e-09227521ba3b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:41.148496 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928adc8c-4f4a-4cf1-9c6e-09227521ba3b None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:41.163605 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:41.176421 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 436/866] 10.222.0.13 () {56 vars in 1135 bytes} [Tue Jun 7 21:50:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:41.178055 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928adc8c-4f4a-4cf1-9c6e-09227521ba3b None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:41.237876 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfb5d18-d04e-4a7c-8006-091d15f3fa0a None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:41.282799 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:41.301192 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4af797b-3f6d-49df-979c-a4ad7d820805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:41.302180 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af797b-3f6d-49df-979c-a4ad7d820805 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:41.302463 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af797b-3f6d-49df-979c-a4ad7d820805 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:41.302735 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af797b-3f6d-49df-979c-a4ad7d820805 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:41.303501 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4af797b-3f6d-49df-979c-a4ad7d820805 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:41.303743 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928adc8c-4f4a-4cf1-9c6e-09227521ba3b None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:41.303856 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4af797b-3f6d-49df-979c-a4ad7d820805 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:41.305171 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4af797b-3f6d-49df-979c-a4ad7d820805 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:41.321344 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 431/867] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:41.326401 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4af797b-3f6d-49df-979c-a4ad7d820805 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:41.348620 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a61a1e-f2e2-4b5a-90af-ff1bd343686f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:41.349584 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a61a1e-f2e2-4b5a-90af-ff1bd343686f None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:41.350061 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a61a1e-f2e2-4b5a-90af-ff1bd343686f None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:41.350341 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a61a1e-f2e2-4b5a-90af-ff1bd343686f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:41.430665 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a61a1e-f2e2-4b5a-90af-ff1bd343686f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:41.432145 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a61a1e-f2e2-4b5a-90af-ff1bd343686f None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:41.465432 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4af797b-3f6d-49df-979c-a4ad7d820805 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:41.479585 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:41.514514 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c6b0f2-be71-4169-90b4-2f1728f810d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:41.515009 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6b0f2-be71-4169-90b4-2f1728f810d8 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:41.515296 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6b0f2-be71-4169-90b4-2f1728f810d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:41.515617 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6b0f2-be71-4169-90b4-2f1728f810d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:41.523145 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c6b0f2-be71-4169-90b4-2f1728f810d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:41.524631 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c6b0f2-be71-4169-90b4-2f1728f810d8 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:41.538424 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:41.539218 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-57c6b0f2-be71-4169-90b4-2f1728f810d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:41.539218 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:41.539218 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:41.539218 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:41.539218 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:41.539218 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:41.539218 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:41.539218 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:41.539218 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:41.539218 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:41.539218 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:41.539218 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:41.539218 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 21:50:41.539218 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 21:50:41.539218 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 21:50:41.540110 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 21:50:41.540110 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:41.540110 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:41.540110 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:41.540110 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:41.540110 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:41.540110 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:41.540110 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:41.540110 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:41.540110 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:41.540110 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:41.540110 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:41.540110 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:41.540110 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:41.540110 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:41.540110 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:41.540110 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:41.541378 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 21:50:41.541378 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 21:50:41.541378 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 21:50:41.541378 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 21:50:41.541378 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 21:50:41.541378 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 21:50:41.541378 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:41.541378 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:41.541378 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 432/868] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:50:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:50:41.546037 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c6b0f2-be71-4169-90b4-2f1728f810d8 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:41.575469 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 437/869] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:50:41 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 07 21:50:41.576644 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a61a1e-f2e2-4b5a-90af-ff1bd343686f None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:41.680506 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c6b0f2-be71-4169-90b4-2f1728f810d8 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:41.689926 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a61a1e-f2e2-4b5a-90af-ff1bd343686f None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:41.725441 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d42e7d-6d56-4d8a-b65d-a4539673300e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:41.726522 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d42e7d-6d56-4d8a-b65d-a4539673300e None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:41.726977 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d42e7d-6d56-4d8a-b65d-a4539673300e None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:41.727564 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d42e7d-6d56-4d8a-b65d-a4539673300e None admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:41.728510 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d42e7d-6d56-4d8a-b65d-a4539673300e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:41.728981 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d42e7d-6d56-4d8a-b65d-a4539673300e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:41.730783 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d42e7d-6d56-4d8a-b65d-a4539673300e None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:41.731096 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded3bbc6-5674-4163-baa4-0bb259d38358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:41.732014 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded3bbc6-5674-4163-baa4-0bb259d38358 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:41.732389 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded3bbc6-5674-4163-baa4-0bb259d38358 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:41.732757 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded3bbc6-5674-4163-baa4-0bb259d38358 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:41.743999 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded3bbc6-5674-4163-baa4-0bb259d38358 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:41.745609 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded3bbc6-5674-4163-baa4-0bb259d38358 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:41.746345 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 433/870] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jun 7 21:50:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:50:41.751778 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d42e7d-6d56-4d8a-b65d-a4539673300e None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:41.774968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-ded3bbc6-5674-4163-baa4-0bb259d38358 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:41.774968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:41.774968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:41.774968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:41.774968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:41.774968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:41.774968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:41.774968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:41.774968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:41.774968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:41.774968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:41.774968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:41.774968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:41.774968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:41.774968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:41.776158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:41.776158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:41.776158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:41.776158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:41.776158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:41.776158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:41.776158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:41.776158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:41.776158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:41.776158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:41.776158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:41.776158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:41.776158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:41.776158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:41.776158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:41.776158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:41.776158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:41.777078 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:41.777078 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:41.777078 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:41.777078 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:41.778013 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 438/871] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:50:41.785837 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded3bbc6-5674-4163-baa4-0bb259d38358 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:41.788903 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:41.835601 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:41.883982 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d42e7d-6d56-4d8a-b65d-a4539673300e None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:41.899136 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded3bbc6-5674-4163-baa4-0bb259d38358 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:41.923994 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae61a963-331e-4c73-a253-a95fe2eb3e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:41.925141 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae61a963-331e-4c73-a253-a95fe2eb3e0f None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:41.925422 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae61a963-331e-4c73-a253-a95fe2eb3e0f None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:41.925694 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae61a963-331e-4c73-a253-a95fe2eb3e0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:41.926591 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae61a963-331e-4c73-a253-a95fe2eb3e0f None admin] RBAC: Adding query filter params (role.id=2c7f73c184d0406bbf2899465953342b, scope.project.id=0219dbda54274fdc9bdccbd960c30c6b, user.id=86619a80df8841a3ba7d8471ac38056a) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:41.926905 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae61a963-331e-4c73-a253-a95fe2eb3e0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:41.928501 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae61a963-331e-4c73-a253-a95fe2eb3e0f None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:41.935375 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 434/872] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:50:41 2022] GET /identity/v3/role_assignments?user.id=86619a80df8841a3ba7d8471ac38056a&scope.project.id=0219dbda54274fdc9bdccbd960c30c6b&role.id=2c7f73c184d0406bbf2899465953342b => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:41.941514 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee9aee8-774a-4457-9330-9a050164e382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:41.942474 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9aee8-774a-4457-9330-9a050164e382 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:41.942908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9aee8-774a-4457-9330-9a050164e382 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:41.943378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9aee8-774a-4457-9330-9a050164e382 None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:41.944291 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee9aee8-774a-4457-9330-9a050164e382 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:41.944754 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee9aee8-774a-4457-9330-9a050164e382 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:41.946348 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee9aee8-774a-4457-9330-9a050164e382 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:41.961208 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 439/873] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:50:41.967210 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae61a963-331e-4c73-a253-a95fe2eb3e0f None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:42.074398 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:42.085639 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae61a963-331e-4c73-a253-a95fe2eb3e0f None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:42.126149 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5226df-0e5e-45e6-83ac-61a39b0cdcef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:42.127419 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5226df-0e5e-45e6-83ac-61a39b0cdcef None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:42.127902 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5226df-0e5e-45e6-83ac-61a39b0cdcef None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:42.128331 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5226df-0e5e-45e6-83ac-61a39b0cdcef None admin] PATH_INFO: `/v3/projects/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:42.136152 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5226df-0e5e-45e6-83ac-61a39b0cdcef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:42.137894 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5226df-0e5e-45e6-83ac-61a39b0cdcef None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:42.153072 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-8a5226df-0e5e-45e6-83ac-61a39b0cdcef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:42.153072 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:42.153072 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:42.153072 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:42.153072 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:42.153072 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:42.153072 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:42.153072 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:42.153072 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:42.153072 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:42.153072 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:42.153072 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:42.153072 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 21:50:42.153072 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 21:50:42.153072 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 21:50:42.154191 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 21:50:42.154191 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:42.154191 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:42.154191 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:42.154191 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:42.154191 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:42.154191 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:42.154191 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:42.154191 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:42.154191 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:42.154191 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:42.154191 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:42.154191 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:42.154191 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:42.154191 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:42.154191 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:42.154191 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:42.155133 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 21:50:42.155133 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 21:50:42.155133 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 21:50:42.155133 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 21:50:42.155133 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 21:50:42.155133 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 21:50:42.155133 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:42.155133 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:42.155982 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 435/874] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:50:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:42.163544 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee9aee8-774a-4457-9330-9a050164e382 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:42.272535 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:42.276938 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee9aee8-774a-4457-9330-9a050164e382 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:42.316511 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbadf8f-92c9-45fa-b444-76ba74093f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:42.317548 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbadf8f-92c9-45fa-b444-76ba74093f8b None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:42.318022 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbadf8f-92c9-45fa-b444-76ba74093f8b None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:42.318478 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbadf8f-92c9-45fa-b444-76ba74093f8b None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:42.319511 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbadf8f-92c9-45fa-b444-76ba74093f8b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:42.320041 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbadf8f-92c9-45fa-b444-76ba74093f8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:42.321700 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbadf8f-92c9-45fa-b444-76ba74093f8b None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:42.335856 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 440/875] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jun 7 21:50:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:42.341953 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5226df-0e5e-45e6-83ac-61a39b0cdcef None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:42.450353 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5226df-0e5e-45e6-83ac-61a39b0cdcef None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:42.488023 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6196cf46-316f-4569-8db8-8f7384692364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:42.488713 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196cf46-316f-4569-8db8-8f7384692364 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:42.489053 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196cf46-316f-4569-8db8-8f7384692364 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:42.489340 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196cf46-316f-4569-8db8-8f7384692364 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:42.490212 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6196cf46-316f-4569-8db8-8f7384692364 None admin] RBAC: Adding query filter params (role.id=6b483758dbd44eb4a78df2ce4603d601, scope.project.id=0219dbda54274fdc9bdccbd960c30c6b, user.id=7a1261563b6d4e50a4ffeea6a32bdb01) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:42.490529 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6196cf46-316f-4569-8db8-8f7384692364 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:42.492145 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6196cf46-316f-4569-8db8-8f7384692364 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:42.499744 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 436/876] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:50:42 2022] GET /identity/v3/role_assignments?user.id=7a1261563b6d4e50a4ffeea6a32bdb01&scope.project.id=0219dbda54274fdc9bdccbd960c30c6b&role.id=6b483758dbd44eb4a78df2ce4603d601 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:42.870326 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d0acd-7141-436a-8ea7-ed8ffb0b4b07 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:42.870721 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d0acd-7141-436a-8ea7-ed8ffb0b4b07 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:42.870908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d0acd-7141-436a-8ea7-ed8ffb0b4b07 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:42.872593 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 441/877] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:42.878358 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c7825-59e5-4d9c-a32b-9579cb017579 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:42.878643 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c7825-59e5-4d9c-a32b-9579cb017579 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:42.878929 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c7825-59e5-4d9c-a32b-9579cb017579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:42.897135 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-c71c7825-59e5-4d9c-a32b-9579cb017579 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:42.898409 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-c71c7825-59e5-4d9c-a32b-9579cb017579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:42.910308 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:42.972959 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 437/878] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:42 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:50:42.995905 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202d0acd-7141-436a-8ea7-ed8ffb0b4b07 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:43.115401 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202d0acd-7141-436a-8ea7-ed8ffb0b4b07 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:43.152576 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e4fac2-c61e-4583-99c5-8dbc7b23260a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:43.153419 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4fac2-c61e-4583-99c5-8dbc7b23260a None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:43.153706 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4fac2-c61e-4583-99c5-8dbc7b23260a None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:43.153983 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4fac2-c61e-4583-99c5-8dbc7b23260a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:43.154662 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e4fac2-c61e-4583-99c5-8dbc7b23260a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:43.156122 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e4fac2-c61e-4583-99c5-8dbc7b23260a None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:43.170110 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-71e4fac2-c61e-4583-99c5-8dbc7b23260a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 21:50:43.170110 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:43.170110 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 21:50:43.170110 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 21:50:43.170110 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 21:50:43.170110 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 21:50:43.170110 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 21:50:43.170110 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:50:43.170110 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 21:50:43.170110 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:43.170110 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 21:50:43.170110 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:43.170110 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:43.170110 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:43.170110 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:43.170110 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:43.171677 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:43.171677 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:43.171677 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:43.171677 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:43.171677 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:43.171677 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:43.171677 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:43.171677 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 21:50:43.171677 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 21:50:43.171677 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 21:50:43.171677 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 21:50:43.171677 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:43.171677 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:43.171677 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:43.171677 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:43.171677 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:43.172531 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:43.172531 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:43.172531 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:43.172531 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:43.172531 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:43.172531 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:43.172531 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:43.172531 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:43.172531 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:43.172531 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:43.172531 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:43.172531 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 21:50:43.172531 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:50:43.172531 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 21:50:43.172531 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:43.172531 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 442/879] 10.222.0.13 () {56 vars in 1132 bytes} [Tue Jun 7 21:50:42 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:43.180460 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71c7825-59e5-4d9c-a32b-9579cb017579 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:43.293129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71c7825-59e5-4d9c-a32b-9579cb017579 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:43.331963 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6b9f3-224f-4cec-86ed-209aaca66467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:43.332633 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6b9f3-224f-4cec-86ed-209aaca66467 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:43.332916 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6b9f3-224f-4cec-86ed-209aaca66467 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:43.333195 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6b9f3-224f-4cec-86ed-209aaca66467 None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:43.333905 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c6b9f3-224f-4cec-86ed-209aaca66467 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:43.334207 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c6b9f3-224f-4cec-86ed-209aaca66467 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:43.335658 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c6b9f3-224f-4cec-86ed-209aaca66467 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:43.341063 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 438/880] 10.222.0.13 () {56 vars in 1126 bytes} [Tue Jun 7 21:50:43 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:43.347452 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e4fac2-c61e-4583-99c5-8dbc7b23260a None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:43.379012 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:43.467179 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e4fac2-c61e-4583-99c5-8dbc7b23260a None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:43.496629 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95471354-3f91-40e3-bef5-85c8ccaed134 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:43.496976 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95471354-3f91-40e3-bef5-85c8ccaed134 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:43.497190 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95471354-3f91-40e3-bef5-85c8ccaed134 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:43.499410 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 439/881] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:43.504214 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf0d43-9fe9-430d-a55a-ccc645e30d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:43.504667 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64605276-69a3-451e-bbb5-e2f165e16301 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:43.504880 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf0d43-9fe9-430d-a55a-ccc645e30d44 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:43.504998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64605276-69a3-451e-bbb5-e2f165e16301 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:43.505157 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf0d43-9fe9-430d-a55a-ccc645e30d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:43.505252 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64605276-69a3-451e-bbb5-e2f165e16301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:43.505428 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf0d43-9fe9-430d-a55a-ccc645e30d44 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:43.506134 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf0d43-9fe9-430d-a55a-ccc645e30d44 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:43.506438 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf0d43-9fe9-430d-a55a-ccc645e30d44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:43.507926 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf0d43-9fe9-430d-a55a-ccc645e30d44 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:43.513404 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 443/882] 10.222.0.13 () {56 vars in 1126 bytes} [Tue Jun 7 21:50:43 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:43.519387 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf0d43-9fe9-430d-a55a-ccc645e30d44 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:43.524415 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-64605276-69a3-451e-bbb5-e2f165e16301 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:43.525607 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-64605276-69a3-451e-bbb5-e2f165e16301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:43.528470 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:43.603434 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 440/883] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:43 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 21:50:43.614689 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:43.621119 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64605276-69a3-451e-bbb5-e2f165e16301 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:43.631257 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf0d43-9fe9-430d-a55a-ccc645e30d44 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:43.666065 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8782b857-9f53-44e0-9c02-302dd80caa5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:43.667073 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782b857-9f53-44e0-9c02-302dd80caa5f None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:43.667585 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782b857-9f53-44e0-9c02-302dd80caa5f None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:43.668105 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782b857-9f53-44e0-9c02-302dd80caa5f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:43.669162 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8782b857-9f53-44e0-9c02-302dd80caa5f None admin] RBAC: Adding query filter params (interface=public, service_id=d855a2d518374d4a8031fec48079c916, region_id=RegionOne) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:43.669667 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8782b857-9f53-44e0-9c02-302dd80caa5f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:43.671281 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8782b857-9f53-44e0-9c02-302dd80caa5f None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:43.678128 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 444/884] 10.222.0.13 () {56 vars in 1251 bytes} [Tue Jun 7 21:50:43 2022] GET /identity/v3/endpoints?service_id=d855a2d518374d4a8031fec48079c916&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:43.687362 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8782b857-9f53-44e0-9c02-302dd80caa5f None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:43.752038 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64605276-69a3-451e-bbb5-e2f165e16301 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:43.803562 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a47f8-3137-4979-a24e-bb5b8d19d3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:43.804393 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a47f8-3137-4979-a24e-bb5b8d19d3c9 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:43.804675 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a47f8-3137-4979-a24e-bb5b8d19d3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:43.804953 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a47f8-3137-4979-a24e-bb5b8d19d3c9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:43.805612 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190a47f8-3137-4979-a24e-bb5b8d19d3c9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:43.807030 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190a47f8-3137-4979-a24e-bb5b8d19d3c9 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:43.818487 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8782b857-9f53-44e0-9c02-302dd80caa5f None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:43.819860 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-190a47f8-3137-4979-a24e-bb5b8d19d3c9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 21:50:43.819860 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:43.819860 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 21:50:43.819860 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 21:50:43.819860 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 21:50:43.819860 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 21:50:43.819860 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 21:50:43.819860 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:50:43.819860 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 21:50:43.819860 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:43.819860 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 21:50:43.819860 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:43.819860 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:43.819860 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:43.819860 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:43.819860 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:43.819860 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:43.820867 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:43.820867 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:43.820867 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:43.820867 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:43.820867 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:43.820867 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:43.820867 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 21:50:43.820867 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 21:50:43.820867 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 21:50:43.820867 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 21:50:43.820867 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:43.820867 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:43.820867 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:43.820867 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:43.820867 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:43.820867 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:43.821736 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:43.821736 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:43.821736 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:43.821736 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:43.821736 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:43.821736 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:43.821736 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:43.821736 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:43.821736 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:43.821736 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:43.821736 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 21:50:43.821736 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:50:43.821736 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 21:50:43.821736 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:43.821736 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 441/885] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Jun 7 21:50:43 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:50:43.827223 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a47f8-3137-4979-a24e-bb5b8d19d3c9 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:43.831883 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:43.859651 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a53e6-c0f5-48a4-a6a9-b8be1f7afd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:43.860469 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a53e6-c0f5-48a4-a6a9-b8be1f7afd79 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:43.860886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a53e6-c0f5-48a4-a6a9-b8be1f7afd79 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:43.861912 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a53e6-c0f5-48a4-a6a9-b8be1f7afd79 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:43.862753 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223a53e6-c0f5-48a4-a6a9-b8be1f7afd79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:43.864452 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223a53e6-c0f5-48a4-a6a9-b8be1f7afd79 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:43.872247 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 445/886] 10.222.0.13 () {56 vars in 1087 bytes} [Tue Jun 7 21:50:43 2022] GET /identity/v3/services => generated 1268 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 21:50:43.951176 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a47f8-3137-4979-a24e-bb5b8d19d3c9 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:43.992620 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d793d2b1-59e2-4a50-abb2-bd6a7d334ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:43.993606 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793d2b1-59e2-4a50-abb2-bd6a7d334ab0 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:43.994071 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793d2b1-59e2-4a50-abb2-bd6a7d334ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:43.994516 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793d2b1-59e2-4a50-abb2-bd6a7d334ab0 None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:43.995419 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d793d2b1-59e2-4a50-abb2-bd6a7d334ab0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:43.995880 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d793d2b1-59e2-4a50-abb2-bd6a7d334ab0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:43.997464 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d793d2b1-59e2-4a50-abb2-bd6a7d334ab0 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:44.009178 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 442/887] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Jun 7 21:50:43 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:50:44.010824 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a53e6-c0f5-48a4-a6a9-b8be1f7afd79 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:44.029900 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:44.129300 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a53e6-c0f5-48a4-a6a9-b8be1f7afd79 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:44.166888 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4b85b9-f8c5-41fe-b350-d2a8968a396f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:44.167602 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b85b9-f8c5-41fe-b350-d2a8968a396f None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:44.167881 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b85b9-f8c5-41fe-b350-d2a8968a396f None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:44.168152 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b85b9-f8c5-41fe-b350-d2a8968a396f None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:44.168876 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4b85b9-f8c5-41fe-b350-d2a8968a396f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:44.169181 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4b85b9-f8c5-41fe-b350-d2a8968a396f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:44.170586 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4b85b9-f8c5-41fe-b350-d2a8968a396f None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:44.177433 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 446/888] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Jun 7 21:50:44 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:44.183630 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d793d2b1-59e2-4a50-abb2-bd6a7d334ab0 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:44.256859 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:44.299336 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d793d2b1-59e2-4a50-abb2-bd6a7d334ab0 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:44.354165 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9233e1d-d73d-4075-9c9c-020f643b8456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:44.354165 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9233e1d-d73d-4075-9c9c-020f643b8456 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:44.354165 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9233e1d-d73d-4075-9c9c-020f643b8456 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:44.354165 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9233e1d-d73d-4075-9c9c-020f643b8456 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:44.354599 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9233e1d-d73d-4075-9c9c-020f643b8456 None admin] RBAC: Adding query filter params (interface=public, service_id=580163013fc947408a3f579b6afa1279, region_id=RegionOne) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:44.354599 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9233e1d-d73d-4075-9c9c-020f643b8456 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:44.354599 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9233e1d-d73d-4075-9c9c-020f643b8456 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:44.370441 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 443/889] 10.222.0.13 () {56 vars in 1251 bytes} [Tue Jun 7 21:50:44 2022] GET /identity/v3/endpoints?service_id=580163013fc947408a3f579b6afa1279&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:44.372444 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4b85b9-f8c5-41fe-b350-d2a8968a396f None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:44.508646 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a281f-efa3-42d9-822d-585c3b2b39c5 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:44.508646 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a281f-efa3-42d9-822d-585c3b2b39c5 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:44.508646 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a281f-efa3-42d9-822d-585c3b2b39c5 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:44.531356 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 444/890] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:44.531356 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e6e5c-9217-4472-8eca-e31120919844 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:44.531356 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e6e5c-9217-4472-8eca-e31120919844 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:44.531356 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e6e5c-9217-4472-8eca-e31120919844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:44.531777 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:44.538426 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4b85b9-f8c5-41fe-b350-d2a8968a396f None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:44.575795 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7952ea08-33c5-461f-bfaf-e685c70d815f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:44.576274 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7952ea08-33c5-461f-bfaf-e685c70d815f None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:44.576557 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7952ea08-33c5-461f-bfaf-e685c70d815f None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:44.576829 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7952ea08-33c5-461f-bfaf-e685c70d815f None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:44.577450 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-067e6e5c-9217-4472-8eca-e31120919844 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:44.577566 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7952ea08-33c5-461f-bfaf-e685c70d815f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:44.578858 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7952ea08-33c5-461f-bfaf-e685c70d815f None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:44.579008 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-067e6e5c-9217-4472-8eca-e31120919844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:44.586829 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 447/891] 10.222.0.13 () {56 vars in 1087 bytes} [Tue Jun 7 21:50:44 2022] GET /identity/v3/services => generated 1268 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 21:50:44.669843 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 445/892] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:44 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:50:44.702733 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7952ea08-33c5-461f-bfaf-e685c70d815f None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:44.851828 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7952ea08-33c5-461f-bfaf-e685c70d815f None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:44.896101 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3507de18-e7e3-405c-aa34-262f6111692f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:44.896923 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507de18-e7e3-405c-aa34-262f6111692f None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:44.897204 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507de18-e7e3-405c-aa34-262f6111692f None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:44.897476 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507de18-e7e3-405c-aa34-262f6111692f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:44.910960 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3507de18-e7e3-405c-aa34-262f6111692f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:50:44.922322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-3507de18-e7e3-405c-aa34-262f6111692f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 21:50:44.922322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:44.922322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:44.922322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:44.922322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:44.922322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:44.922322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:44.922322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:44.922322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:44.922322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:44.922322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:44.922322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:44.922322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:50:44.922322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:50:44.922322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:50:44.922322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:50:44.923987 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:50:44.923987 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:50:44.923987 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:50:44.923987 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:50:44.923987 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:44.923987 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:44.923987 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:44.923987 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:44.923987 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:44.923987 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:44.923987 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:44.923987 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:44.923987 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:44.923987 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:44.923987 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:44.923987 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:44.924960 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:44.924960 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:44.924960 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:44.924960 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:44.924960 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:50:44.924960 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:50:44.924960 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:50:44.924960 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:50:44.924960 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:50:44.924960 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:50:44.924960 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 21:50:44.924960 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:44.925667 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 448/893] 10.222.0.13 () {56 vars in 1120 bytes} [Tue Jun 7 21:50:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:44.934612 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e6e5c-9217-4472-8eca-e31120919844 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:45.011837 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:45.054159 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e6e5c-9217-4472-8eca-e31120919844 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:45.087692 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e5213-a522-4fdb-a4b3-2c7c32419ef9 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:45.088042 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e5213-a522-4fdb-a4b3-2c7c32419ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:45.088231 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e5213-a522-4fdb-a4b3-2c7c32419ef9 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:45.090569 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 449/894] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:45.098054 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351bb076-70aa-4c9d-941b-1b5a2e01cacd None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:45.098649 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351bb076-70aa-4c9d-941b-1b5a2e01cacd None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:45.099127 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351bb076-70aa-4c9d-941b-1b5a2e01cacd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:45.099446 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8079df16-db95-47a5-9960-8e7b7b1c4f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:45.100187 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079df16-db95-47a5-9960-8e7b7b1c4f58 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:45.100716 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079df16-db95-47a5-9960-8e7b7b1c4f58 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:45.101276 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079df16-db95-47a5-9960-8e7b7b1c4f58 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:45.102606 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8079df16-db95-47a5-9960-8e7b7b1c4f58 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:45.103010 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8079df16-db95-47a5-9960-8e7b7b1c4f58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:45.105193 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8079df16-db95-47a5-9960-8e7b7b1c4f58 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:45.118802 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 446/895] 10.222.0.13 () {56 vars in 1115 bytes} [Tue Jun 7 21:50:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:45.119814 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:45.120972 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-351bb076-70aa-4c9d-941b-1b5a2e01cacd None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:45.122657 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-351bb076-70aa-4c9d-941b-1b5a2e01cacd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:45.124876 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8079df16-db95-47a5-9960-8e7b7b1c4f58 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:45.207420 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 450/896] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:45 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 21:50:45.227238 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351bb076-70aa-4c9d-941b-1b5a2e01cacd None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:45.270232 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8079df16-db95-47a5-9960-8e7b7b1c4f58 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:45.299477 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:45.314496 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58da6e9-c7cf-4466-bb85-2044e9514c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:45.315441 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58da6e9-c7cf-4466-bb85-2044e9514c9e None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:45.315797 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58da6e9-c7cf-4466-bb85-2044e9514c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:45.316141 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58da6e9-c7cf-4466-bb85-2044e9514c9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:45.332102 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58da6e9-c7cf-4466-bb85-2044e9514c9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:45.333997 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58da6e9-c7cf-4466-bb85-2044e9514c9e None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:45.356571 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-f58da6e9-c7cf-4466-bb85-2044e9514c9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:45.356571 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:45.356571 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:45.356571 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:45.356571 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:45.356571 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:45.356571 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:45.356571 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:45.356571 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:45.356571 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:45.356571 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:45.356571 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:45.356571 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:45.356571 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:45.356571 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:45.357599 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:45.357599 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:45.357599 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:45.357599 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:45.357599 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:45.357599 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:45.357599 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:45.357599 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:45.357599 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:45.357599 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:45.357599 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:45.357599 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:45.357599 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:45.357599 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:45.357599 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:45.357599 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:45.357599 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:45.358543 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:45.358543 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:45.358543 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:45.358543 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:45.358543 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 447/897] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:50:45.367070 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58da6e9-c7cf-4466-bb85-2044e9514c9e None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:45.378879 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351bb076-70aa-4c9d-941b-1b5a2e01cacd None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:45.420023 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:45.427081 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a3f90d-633d-43c9-9124-7b00ee4b769b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:45.428478 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a3f90d-633d-43c9-9124-7b00ee4b769b None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:45.428983 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a3f90d-633d-43c9-9124-7b00ee4b769b None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:45.429483 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a3f90d-633d-43c9-9124-7b00ee4b769b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:45.430592 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a3f90d-633d-43c9-9124-7b00ee4b769b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:45.433017 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a3f90d-633d-43c9-9124-7b00ee4b769b None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:45.452193 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-b6a3f90d-633d-43c9-9124-7b00ee4b769b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 21:50:45.452193 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:45.452193 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 21:50:45.452193 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 21:50:45.452193 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 21:50:45.452193 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 21:50:45.452193 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 21:50:45.452193 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:50:45.452193 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 21:50:45.452193 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:45.452193 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 21:50:45.452193 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:45.452193 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:45.452193 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:45.452193 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:45.452193 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:45.452193 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:45.453238 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:45.453238 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:45.453238 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:45.453238 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:45.453238 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:45.453238 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:45.453238 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 21:50:45.453238 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 21:50:45.453238 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 21:50:45.453238 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 21:50:45.453238 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:45.453238 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:45.453238 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:45.453238 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:45.453238 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:45.453238 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:45.454079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:45.454079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:45.454079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:45.454079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:45.454079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:45.454079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:45.454079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:45.454079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:45.454079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:45.454079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:45.454079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 21:50:45.454079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:50:45.454079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 21:50:45.454079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:45.454079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 451/898] 10.222.0.13 () {56 vars in 1126 bytes} [Tue Jun 7 21:50:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:45.463652 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a3f90d-633d-43c9-9124-7b00ee4b769b None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:45.504426 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58da6e9-c7cf-4466-bb85-2044e9514c9e None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:45.547076 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5d470a-92a3-4cf9-afc0-37e344b4e39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:45.548022 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d470a-92a3-4cf9-afc0-37e344b4e39c None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:45.548626 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d470a-92a3-4cf9-afc0-37e344b4e39c None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:45.549430 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d470a-92a3-4cf9-afc0-37e344b4e39c None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:45.550634 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5d470a-92a3-4cf9-afc0-37e344b4e39c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:45.551205 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5d470a-92a3-4cf9-afc0-37e344b4e39c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:45.552744 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5d470a-92a3-4cf9-afc0-37e344b4e39c None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:45.575777 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 448/899] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:50:45.577088 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5d470a-92a3-4cf9-afc0-37e344b4e39c None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:45.594286 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a3f90d-633d-43c9-9124-7b00ee4b769b None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:45.622911 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:45.635891 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e8b04-7f0c-450a-aea2-5160b0ee7453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:45.636818 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e8b04-7f0c-450a-aea2-5160b0ee7453 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:45.637280 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e8b04-7f0c-450a-aea2-5160b0ee7453 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:45.637767 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e8b04-7f0c-450a-aea2-5160b0ee7453 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:45.638629 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501e8b04-7f0c-450a-aea2-5160b0ee7453 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:45.639093 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501e8b04-7f0c-450a-aea2-5160b0ee7453 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:45.640767 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501e8b04-7f0c-450a-aea2-5160b0ee7453 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:45.647165 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 452/900] 10.222.0.13 () {56 vars in 1122 bytes} [Tue Jun 7 21:50:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:45.655390 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e8b04-7f0c-450a-aea2-5160b0ee7453 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:45.708904 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5d470a-92a3-4cf9-afc0-37e344b4e39c None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:45.725069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:45.768223 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97efd78b-600f-42de-8f06-b93704c9a326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:45.769337 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97efd78b-600f-42de-8f06-b93704c9a326 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:45.769632 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97efd78b-600f-42de-8f06-b93704c9a326 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:45.769887 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97efd78b-600f-42de-8f06-b93704c9a326 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:45.782204 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97efd78b-600f-42de-8f06-b93704c9a326 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:45.785146 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97efd78b-600f-42de-8f06-b93704c9a326 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:45.803921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-97efd78b-600f-42de-8f06-b93704c9a326 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 21:50:45.803921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:45.803921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:45.803921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:45.803921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:45.803921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:45.803921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:45.803921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:45.803921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:45.803921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:45.803921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:45.803921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:45.803921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 21:50:45.803921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 21:50:45.803921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 21:50:45.803921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 21:50:45.804937 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:45.804937 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:45.804937 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 21:50:45.804937 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:45.804937 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 21:50:45.804937 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:45.804937 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:45.804937 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:45.804937 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:45.804937 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:45.804937 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:45.804937 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:45.804937 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:45.804937 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:45.804937 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:45.804937 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:45.804937 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:45.804937 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:45.805910 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:45.805910 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:45.805910 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 21:50:45.805910 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 21:50:45.805910 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 21:50:45.805910 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 21:50:45.805910 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 21:50:45.805910 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 21:50:45.805910 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 21:50:45.805910 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:45.806449 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e8b04-7f0c-450a-aea2-5160b0ee7453 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:45.813298 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 449/901] 10.222.0.13 () {56 vars in 1117 bytes} [Tue Jun 7 21:50:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:45.814441 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97efd78b-600f-42de-8f06-b93704c9a326 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:45.856834 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae61c4c3-579e-4ec3-aa64-8bd51b5dfad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:45.858092 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae61c4c3-579e-4ec3-aa64-8bd51b5dfad6 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:45.858700 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae61c4c3-579e-4ec3-aa64-8bd51b5dfad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:45.859269 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae61c4c3-579e-4ec3-aa64-8bd51b5dfad6 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:45.861443 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae61c4c3-579e-4ec3-aa64-8bd51b5dfad6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:45.862038 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae61c4c3-579e-4ec3-aa64-8bd51b5dfad6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:45.864461 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae61c4c3-579e-4ec3-aa64-8bd51b5dfad6 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:45.871368 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 453/902] 10.222.0.13 () {56 vars in 1122 bytes} [Tue Jun 7 21:50:45 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:50:45.932556 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:45.935526 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97efd78b-600f-42de-8f06-b93704c9a326 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:45.988880 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aa3d26-70ce-4d63-97aa-ce6369157484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:45.989641 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aa3d26-70ce-4d63-97aa-ce6369157484 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:45.989924 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aa3d26-70ce-4d63-97aa-ce6369157484 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:45.990201 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aa3d26-70ce-4d63-97aa-ce6369157484 None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:45.991065 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aa3d26-70ce-4d63-97aa-ce6369157484 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:45.991406 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aa3d26-70ce-4d63-97aa-ce6369157484 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:45.992892 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aa3d26-70ce-4d63-97aa-ce6369157484 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:46.014064 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 450/903] 10.222.0.13 () {56 vars in 1149 bytes} [Tue Jun 7 21:50:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:46.020647 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae61c4c3-579e-4ec3-aa64-8bd51b5dfad6 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:46.161480 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae61c4c3-579e-4ec3-aa64-8bd51b5dfad6 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:46.186640 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:46.198635 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240bd52f-7a1a-42b0-a82d-9a06eeb07ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:46.199294 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240bd52f-7a1a-42b0-a82d-9a06eeb07ead None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:46.199642 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240bd52f-7a1a-42b0-a82d-9a06eeb07ead None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:46.199926 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240bd52f-7a1a-42b0-a82d-9a06eeb07ead None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:46.273653 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240bd52f-7a1a-42b0-a82d-9a06eeb07ead None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:46.275109 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240bd52f-7a1a-42b0-a82d-9a06eeb07ead None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:46.382229 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 454/904] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:50:46 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 21:50:46.387457 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aa3d26-70ce-4d63-97aa-ce6369157484 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:46.471853 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1cd89a-e4ba-48ca-931d-58d4ecf60ef0 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:46.472437 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1cd89a-e4ba-48ca-931d-58d4ecf60ef0 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:46.472622 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1cd89a-e4ba-48ca-931d-58d4ecf60ef0 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:46.474271 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 455/905] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:46.480646 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc826ef-0912-4119-a025-34f583ee5a8b None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:46.480944 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc826ef-0912-4119-a025-34f583ee5a8b None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:46.481222 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc826ef-0912-4119-a025-34f583ee5a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:46.500988 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-7cc826ef-0912-4119-a025-34f583ee5a8b None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:46.502802 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-7cc826ef-0912-4119-a025-34f583ee5a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:46.504525 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aa3d26-70ce-4d63-97aa-ce6369157484 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:46.523313 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:46.552317 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcba4ce-724b-4ed4-a9ec-91a292ab48d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:46.553646 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcba4ce-724b-4ed4-a9ec-91a292ab48d8 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:46.554215 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcba4ce-724b-4ed4-a9ec-91a292ab48d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:46.554754 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcba4ce-724b-4ed4-a9ec-91a292ab48d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:46.566824 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcba4ce-724b-4ed4-a9ec-91a292ab48d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:46.568635 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcba4ce-724b-4ed4-a9ec-91a292ab48d8 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:46.585890 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 456/906] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:46 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 21:50:46.595426 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-9dcba4ce-724b-4ed4-a9ec-91a292ab48d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:46.595426 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:46.595426 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:46.595426 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:46.595426 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:46.595426 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:46.595426 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:46.595426 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:46.595426 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:46.595426 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:46.595426 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:46.595426 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:46.595426 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:46.595426 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:46.595426 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:46.596597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:46.596597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:46.596597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:46.596597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:46.596597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:46.596597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:46.596597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:46.596597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:46.596597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:46.596597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:46.596597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:46.596597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:46.596597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:46.596597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:46.596597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:46.596597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:46.596597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:46.597769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:46.597769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:46.597769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:46.597769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:46.597769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 451/907] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:46.598049 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc826ef-0912-4119-a025-34f583ee5a8b None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:46.608540 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcba4ce-724b-4ed4-a9ec-91a292ab48d8 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:46.618960 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:46.727018 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcba4ce-724b-4ed4-a9ec-91a292ab48d8 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:46.731101 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc826ef-0912-4119-a025-34f583ee5a8b None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:46.776107 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b72a5-6579-477c-9949-aa04b4ee1f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:46.776924 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b72a5-6579-477c-9949-aa04b4ee1f9f None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:46.777305 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b72a5-6579-477c-9949-aa04b4ee1f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:46.777683 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b72a5-6579-477c-9949-aa04b4ee1f9f None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:46.778508 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8b72a5-6579-477c-9949-aa04b4ee1f9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:46.778915 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8b72a5-6579-477c-9949-aa04b4ee1f9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:46.780432 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8b72a5-6579-477c-9949-aa04b4ee1f9f None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:46.785627 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877a7198-b41d-4b02-ac8c-7e9ab19b6bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:46.786735 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a7198-b41d-4b02-ac8c-7e9ab19b6bec None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:46.787173 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a7198-b41d-4b02-ac8c-7e9ab19b6bec None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:46.787671 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a7198-b41d-4b02-ac8c-7e9ab19b6bec None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:46.788507 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877a7198-b41d-4b02-ac8c-7e9ab19b6bec None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:46.790954 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877a7198-b41d-4b02-ac8c-7e9ab19b6bec None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:46.795653 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 457/908] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:46.806454 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b72a5-6579-477c-9949-aa04b4ee1f9f None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:46.811014 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-877a7198-b41d-4b02-ac8c-7e9ab19b6bec None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 21:50:46.811014 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:46.811014 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 21:50:46.811014 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 21:50:46.811014 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 21:50:46.811014 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 21:50:46.811014 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 21:50:46.811014 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:50:46.811014 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 21:50:46.811014 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:46.811014 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 21:50:46.811014 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:46.811014 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:46.811014 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:46.811014 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:46.811014 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:46.812096 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:46.812096 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:46.812096 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:46.812096 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:46.812096 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:46.812096 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:46.812096 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:46.812096 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 21:50:46.812096 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 21:50:46.812096 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 21:50:46.812096 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 21:50:46.812096 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:46.812096 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:46.812096 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:46.812096 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:46.812096 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:46.812963 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:46.812963 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:46.812963 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:46.812963 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:46.812963 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:46.812963 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:46.812963 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:46.812963 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:46.812963 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:46.812963 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:46.812963 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:46.812963 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 21:50:46.812963 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:50:46.812963 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 21:50:46.812963 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:46.818517 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 452/909] 10.222.0.13 () {56 vars in 1132 bytes} [Tue Jun 7 21:50:46 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:50:46.819772 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877a7198-b41d-4b02-ac8c-7e9ab19b6bec None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:46.825672 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:46.923897 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b72a5-6579-477c-9949-aa04b4ee1f9f None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:46.928843 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:46.961539 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423d253-5fb7-4e02-b46f-eaffba9aac9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:46.962278 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877a7198-b41d-4b02-ac8c-7e9ab19b6bec None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:46.962416 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423d253-5fb7-4e02-b46f-eaffba9aac9b None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:46.962604 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423d253-5fb7-4e02-b46f-eaffba9aac9b None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:46.962880 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423d253-5fb7-4e02-b46f-eaffba9aac9b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:46.970669 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b423d253-5fb7-4e02-b46f-eaffba9aac9b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:46.972509 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b423d253-5fb7-4e02-b46f-eaffba9aac9b None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:46.987365 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-b423d253-5fb7-4e02-b46f-eaffba9aac9b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:46.987365 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:46.987365 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:46.987365 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:46.987365 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:46.987365 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:46.987365 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:46.987365 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:46.987365 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:46.987365 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:46.987365 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:46.987365 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:46.987365 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 21:50:46.987365 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 21:50:46.987365 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 21:50:46.988294 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 21:50:46.988294 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:46.988294 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:46.988294 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:46.988294 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:46.988294 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:46.988294 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:46.988294 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:46.988294 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:46.988294 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:46.988294 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:46.988294 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:46.988294 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:46.988294 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:46.988294 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:46.988294 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:46.988294 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:46.989186 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 21:50:46.989186 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 21:50:46.989186 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 21:50:46.989186 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 21:50:46.989186 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 21:50:46.989186 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 21:50:46.989186 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:46.989186 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:46.993375 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 458/910] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:50:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:46.994489 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423d253-5fb7-4e02-b46f-eaffba9aac9b None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:47.008017 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548890a9-485d-40de-b733-d4a64d4f5e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:47.009566 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548890a9-485d-40de-b733-d4a64d4f5e9b None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:47.010081 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548890a9-485d-40de-b733-d4a64d4f5e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:47.010581 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548890a9-485d-40de-b733-d4a64d4f5e9b None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:47.011712 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548890a9-485d-40de-b733-d4a64d4f5e9b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:47.012593 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548890a9-485d-40de-b733-d4a64d4f5e9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:47.014605 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548890a9-485d-40de-b733-d4a64d4f5e9b None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:47.021928 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 453/911] 10.222.0.13 () {56 vars in 1126 bytes} [Tue Jun 7 21:50:46 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:50:47.028486 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548890a9-485d-40de-b733-d4a64d4f5e9b None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:47.084637 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:47.113189 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423d253-5fb7-4e02-b46f-eaffba9aac9b None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:47.150900 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d939dc3a-88bc-4314-bc7a-3175107079a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:47.151621 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939dc3a-88bc-4314-bc7a-3175107079a7 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:47.151910 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939dc3a-88bc-4314-bc7a-3175107079a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:47.152190 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939dc3a-88bc-4314-bc7a-3175107079a7 None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:47.153035 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d939dc3a-88bc-4314-bc7a-3175107079a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:47.153353 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d939dc3a-88bc-4314-bc7a-3175107079a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:47.154886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d939dc3a-88bc-4314-bc7a-3175107079a7 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:47.164475 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548890a9-485d-40de-b733-d4a64d4f5e9b None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:47.175701 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 459/912] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jun 7 21:50:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:47.177487 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc9e9d-ad68-4b21-996d-052b5612a9a6 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:47.177959 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc9e9d-ad68-4b21-996d-052b5612a9a6 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:47.178338 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc9e9d-ad68-4b21-996d-052b5612a9a6 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:47.179839 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 460/913] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:47.180434 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc9e9d-ad68-4b21-996d-052b5612a9a6 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:47.210140 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb0c20-ddf0-4d77-8fc0-491993fffe16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:47.210935 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb0c20-ddf0-4d77-8fc0-491993fffe16 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:47.211226 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb0c20-ddf0-4d77-8fc0-491993fffe16 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:47.211531 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb0c20-ddf0-4d77-8fc0-491993fffe16 None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:47.212409 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfb0c20-ddf0-4d77-8fc0-491993fffe16 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:47.212718 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfb0c20-ddf0-4d77-8fc0-491993fffe16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:47.214410 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfb0c20-ddf0-4d77-8fc0-491993fffe16 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:47.216144 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:47.221277 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 454/914] 10.222.0.13 () {56 vars in 1126 bytes} [Tue Jun 7 21:50:47 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:50:47.225623 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b552c4-63f4-4c73-a08c-1312d9e30c9f None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:47.225969 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b552c4-63f4-4c73-a08c-1312d9e30c9f None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:47.226262 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b552c4-63f4-4c73-a08c-1312d9e30c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:47.244236 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-18b552c4-63f4-4c73-a08c-1312d9e30c9f None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:47.245346 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-18b552c4-63f4-4c73-a08c-1312d9e30c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:47.298666 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc9e9d-ad68-4b21-996d-052b5612a9a6 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:47.327465 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 455/915] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:47 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:50:47.328480 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b552c4-63f4-4c73-a08c-1312d9e30c9f None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:47.334772 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a244fa2-320c-488d-8d75-f539da78fd41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:47.335456 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a244fa2-320c-488d-8d75-f539da78fd41 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:47.335740 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a244fa2-320c-488d-8d75-f539da78fd41 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:47.336012 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a244fa2-320c-488d-8d75-f539da78fd41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:47.337655 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a244fa2-320c-488d-8d75-f539da78fd41 None admin] RBAC: Adding query filter params (role.id=14b78e4dcfb84633981a9b2f2f06f4fb, scope.project.id=0219dbda54274fdc9bdccbd960c30c6b, user.id=86619a80df8841a3ba7d8471ac38056a) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:47.337971 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a244fa2-320c-488d-8d75-f539da78fd41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:47.339439 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a244fa2-320c-488d-8d75-f539da78fd41 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:47.341589 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:47.351423 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 461/916] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:50:47 2022] GET /identity/v3/role_assignments?user.id=86619a80df8841a3ba7d8471ac38056a&scope.project.id=0219dbda54274fdc9bdccbd960c30c6b&role.id=14b78e4dcfb84633981a9b2f2f06f4fb => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:47.356385 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a244fa2-320c-488d-8d75-f539da78fd41 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:47.467473 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b552c4-63f4-4c73-a08c-1312d9e30c9f None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:47.498008 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a244fa2-320c-488d-8d75-f539da78fd41 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:47.506900 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:47.516359 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db0606-19ff-49ae-827c-bbc955e892a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:47.517829 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db0606-19ff-49ae-827c-bbc955e892a7 None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:47.518205 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db0606-19ff-49ae-827c-bbc955e892a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:47.518605 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db0606-19ff-49ae-827c-bbc955e892a7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:47.519557 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5db0606-19ff-49ae-827c-bbc955e892a7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:47.521183 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5db0606-19ff-49ae-827c-bbc955e892a7 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:47.544447 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04864f83-50c8-4428-a64d-30aa5355edde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:47.545521 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04864f83-50c8-4428-a64d-30aa5355edde None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:47.545937 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04864f83-50c8-4428-a64d-30aa5355edde None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:47.546313 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04864f83-50c8-4428-a64d-30aa5355edde None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:47.547130 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04864f83-50c8-4428-a64d-30aa5355edde None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:47.548166 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 456/917] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Jun 7 21:50:47 2022] POST /identity/v3/endpoints => generated 347 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 21:50:47.548922 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04864f83-50c8-4428-a64d-30aa5355edde None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:47.568443 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-04864f83-50c8-4428-a64d-30aa5355edde None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 21:50:47.568443 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:47.568443 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 21:50:47.568443 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 21:50:47.568443 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 21:50:47.568443 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 21:50:47.568443 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 21:50:47.568443 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:50:47.568443 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 21:50:47.568443 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:47.568443 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 21:50:47.568443 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:47.568443 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:47.568443 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:47.568443 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:47.568443 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:47.568443 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:47.569557 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:47.569557 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:47.569557 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:47.569557 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:47.569557 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:47.569557 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:47.569557 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 21:50:47.569557 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 21:50:47.569557 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 21:50:47.569557 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 21:50:47.569557 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:47.569557 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:47.569557 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:47.569557 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:47.569557 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:47.569557 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:47.570403 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:47.570403 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:47.570403 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:47.570403 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:47.570403 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:47.570403 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:47.570403 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:47.570403 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:47.570403 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:47.570403 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:47.570403 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 21:50:47.570403 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:50:47.570403 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 21:50:47.570403 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:47.570403 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 462/918] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Jun 7 21:50:47 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:47.576579 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db0606-19ff-49ae-827c-bbc955e892a7 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:47.704500 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db0606-19ff-49ae-827c-bbc955e892a7 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:47.745066 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0725171a-774f-49fe-991e-4fea3b4a97cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:47.746116 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725171a-774f-49fe-991e-4fea3b4a97cf None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:47.746604 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725171a-774f-49fe-991e-4fea3b4a97cf None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:47.747080 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725171a-774f-49fe-991e-4fea3b4a97cf None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:47.748151 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0725171a-774f-49fe-991e-4fea3b4a97cf None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:47.748736 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0725171a-774f-49fe-991e-4fea3b4a97cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:47.750428 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0725171a-774f-49fe-991e-4fea3b4a97cf None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:47.757094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 457/919] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Jun 7 21:50:47 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:47.762243 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04864f83-50c8-4428-a64d-30aa5355edde None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:47.842811 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:47.886026 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04864f83-50c8-4428-a64d-30aa5355edde None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:47.922246 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96e3ebe-4615-4547-a894-d4fd54819d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:47.922921 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e3ebe-4615-4547-a894-d4fd54819d29 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:47.923766 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e3ebe-4615-4547-a894-d4fd54819d29 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:47.924043 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e3ebe-4615-4547-a894-d4fd54819d29 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:47.924759 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96e3ebe-4615-4547-a894-d4fd54819d29 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:47.925067 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96e3ebe-4615-4547-a894-d4fd54819d29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:47.926448 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96e3ebe-4615-4547-a894-d4fd54819d29 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:47.931969 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 463/920] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Jun 7 21:50:47 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:50:47.937198 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0725171a-774f-49fe-991e-4fea3b4a97cf None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:47.991714 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:48.053911 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0725171a-774f-49fe-991e-4fea3b4a97cf None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:48.089150 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed7819f-fcad-40b2-829e-19e7e4445a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:48.089819 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7819f-fcad-40b2-829e-19e7e4445a4d None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:48.090106 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7819f-fcad-40b2-829e-19e7e4445a4d None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:48.090384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7819f-fcad-40b2-829e-19e7e4445a4d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:48.091195 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed7819f-fcad-40b2-829e-19e7e4445a4d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:48.092773 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed7819f-fcad-40b2-829e-19e7e4445a4d None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:48.120535 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 458/921] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Jun 7 21:50:47 2022] POST /identity/v3/endpoints => generated 346 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 21:50:48.364837 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762a543-7eaf-4dae-92fe-473c8f2ce8b7 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:48.365335 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762a543-7eaf-4dae-92fe-473c8f2ce8b7 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:48.365335 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762a543-7eaf-4dae-92fe-473c8f2ce8b7 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:48.366834 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 464/922] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:48.372524 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909a417-4dfb-4946-bd5d-5a4d2e211857 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:48.373082 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909a417-4dfb-4946-bd5d-5a4d2e211857 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:48.373569 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909a417-4dfb-4946-bd5d-5a4d2e211857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:48.391095 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-8909a417-4dfb-4946-bd5d-5a4d2e211857 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:48.392712 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-8909a417-4dfb-4946-bd5d-5a4d2e211857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:48.471460 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 459/923] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:48 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:50:48.491674 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0762a543-7eaf-4dae-92fe-473c8f2ce8b7 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:48.609157 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0762a543-7eaf-4dae-92fe-473c8f2ce8b7 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:48.638934 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:48.648518 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22d79f8-2732-4ce4-9ccf-61a1e15420e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:48.649288 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d79f8-2732-4ce4-9ccf-61a1e15420e9 None admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:48.649789 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d79f8-2732-4ce4-9ccf-61a1e15420e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:48.650068 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d79f8-2732-4ce4-9ccf-61a1e15420e9 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:48.650796 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22d79f8-2732-4ce4-9ccf-61a1e15420e9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:48.652215 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22d79f8-2732-4ce4-9ccf-61a1e15420e9 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:48.675542 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 465/924] 10.222.0.13 () {60 vars in 1141 bytes} [Tue Jun 7 21:50:48 2022] POST /identity/v3/services => generated 241 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 21:50:49.784974 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acbece1-820e-44a3-8d51-fff9a7fdd626 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:49.785515 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acbece1-820e-44a3-8d51-fff9a7fdd626 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:49.785515 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acbece1-820e-44a3-8d51-fff9a7fdd626 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:49.787099 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 460/925] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:49.792559 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b5165-de56-4817-9f4b-a112b6219fc2 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:49.792962 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b5165-de56-4817-9f4b-a112b6219fc2 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:49.793147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b5165-de56-4817-9f4b-a112b6219fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:49.809956 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-d80b5165-de56-4817-9f4b-a112b6219fc2 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:49.811134 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d80b5165-de56-4817-9f4b-a112b6219fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:49.890735 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 466/926] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 21:50:49.914532 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acbece1-820e-44a3-8d51-fff9a7fdd626 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:49.943357 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:49.964482 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a2022-80a6-400e-b84d-561e1d43251c None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:49.965138 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a2022-80a6-400e-b84d-561e1d43251c None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:49.965618 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a2022-80a6-400e-b84d-561e1d43251c None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:49.967466 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 467/927] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 21:50:49.973533 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34360698-4557-4910-9531-f23996c9f4e1 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:49.974099 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34360698-4557-4910-9531-f23996c9f4e1 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:49.974581 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34360698-4557-4910-9531-f23996c9f4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:49.993512 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-34360698-4557-4910-9531-f23996c9f4e1 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:49.995290 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-34360698-4557-4910-9531-f23996c9f4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:50.029494 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acbece1-820e-44a3-8d51-fff9a7fdd626 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:50.065653 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78680c9a-cff8-4b17-bfcb-0f3cb5c67557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:50.066625 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78680c9a-cff8-4b17-bfcb-0f3cb5c67557 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:50.067009 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78680c9a-cff8-4b17-bfcb-0f3cb5c67557 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:50.067414 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78680c9a-cff8-4b17-bfcb-0f3cb5c67557 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:50.074386 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 468/928] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 21:50:50.083287 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78680c9a-cff8-4b17-bfcb-0f3cb5c67557 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:50:50.091697 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-78680c9a-cff8-4b17-bfcb-0f3cb5c67557 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 21:50:50.091697 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:50.091697 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:50.091697 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:50.091697 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:50.091697 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:50.091697 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:50.091697 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:50.091697 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:50.091697 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:50.091697 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:50.091697 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:50.091697 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:50:50.091697 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:50:50.091697 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:50:50.091697 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:50:50.092630 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:50:50.092630 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:50:50.092630 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:50:50.092630 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:50:50.092630 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:50.092630 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:50.092630 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:50.092630 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:50.092630 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:50.092630 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:50.092630 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:50.092630 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:50.092630 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:50.092630 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:50.092630 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:50.092630 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:50.093476 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:50.093476 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:50.093476 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:50.093476 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:50.093476 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:50:50.093476 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:50:50.093476 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:50:50.093476 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:50:50.093476 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:50:50.093476 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:50:50.093476 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 21:50:50.093476 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:50.096523 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 461/929] 10.222.0.13 () {56 vars in 1120 bytes} [Tue Jun 7 21:50:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:50.098019 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34360698-4557-4910-9531-f23996c9f4e1 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:50.103661 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78680c9a-cff8-4b17-bfcb-0f3cb5c67557 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:50.147529 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:50.213130 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:50.228737 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34360698-4557-4910-9531-f23996c9f4e1 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:50.237658 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78680c9a-cff8-4b17-bfcb-0f3cb5c67557 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:50.270170 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453c9167-cb0c-44cd-957b-1ee98011854e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:50.271251 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c9167-cb0c-44cd-957b-1ee98011854e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:50.271583 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c9167-cb0c-44cd-957b-1ee98011854e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:50.271916 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c9167-cb0c-44cd-957b-1ee98011854e None admin] PATH_INFO: `/v3/services/compute` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:50.273082 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453c9167-cb0c-44cd-957b-1ee98011854e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:50.274993 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453c9167-cb0c-44cd-957b-1ee98011854e None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:50.284537 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3380de7c-a9f6-4c21-b289-7fd14cd7ae1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:50.285412 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380de7c-a9f6-4c21-b289-7fd14cd7ae1e None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:50.285844 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380de7c-a9f6-4c21-b289-7fd14cd7ae1e None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:50.286265 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380de7c-a9f6-4c21-b289-7fd14cd7ae1e None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:50.287169 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3380de7c-a9f6-4c21-b289-7fd14cd7ae1e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:50.287694 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3380de7c-a9f6-4c21-b289-7fd14cd7ae1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:50.288776 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-453c9167-cb0c-44cd-957b-1ee98011854e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 21:50:50.288776 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:50.288776 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 21:50:50.288776 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 21:50:50.288776 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 21:50:50.288776 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 21:50:50.288776 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 21:50:50.288776 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:50:50.288776 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 21:50:50.288776 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:50.288776 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 21:50:50.288776 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:50.288776 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:50.288776 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:50.288776 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:50.288776 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:50.288776 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:50.289967 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:50.289967 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:50.289967 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:50.289967 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:50.289967 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:50.289967 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:50.289967 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 21:50:50.289967 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 21:50:50.289967 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 21:50:50.289967 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 21:50:50.289967 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:50.289967 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:50.289967 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:50.289967 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:50.289967 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:50.289967 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:50.290840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:50.290840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:50.290840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:50.290840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:50.290840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:50.290840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:50.290840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:50.290840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:50.290840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:50.290840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:50.290840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 21:50:50.290840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:50:50.290840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 21:50:50.290840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:50.290840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 469/930] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:50:50 2022] GET /identity/v3/services/compute => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:50.292118 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3380de7c-a9f6-4c21-b289-7fd14cd7ae1e None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:50.296288 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453c9167-cb0c-44cd-957b-1ee98011854e None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:50.306139 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 462/931] 10.222.0.13 () {56 vars in 1115 bytes} [Tue Jun 7 21:50:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:50.307589 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3380de7c-a9f6-4c21-b289-7fd14cd7ae1e None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:50.408924 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453c9167-cb0c-44cd-957b-1ee98011854e None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:50.412978 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3380de7c-a9f6-4c21-b289-7fd14cd7ae1e None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:50.417624 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:50.450013 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4806999a-4050-4eea-9852-1adfc1519948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:50.451291 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4806999a-4050-4eea-9852-1adfc1519948 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:50.451911 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4806999a-4050-4eea-9852-1adfc1519948 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:50.452441 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4806999a-4050-4eea-9852-1adfc1519948 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:50.453481 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4806999a-4050-4eea-9852-1adfc1519948 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:50.453776 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a939b-e82e-4e09-8a2f-da312f811e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:50.454268 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4806999a-4050-4eea-9852-1adfc1519948 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:50.454802 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a939b-e82e-4e09-8a2f-da312f811e8e None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:50.455094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a939b-e82e-4e09-8a2f-da312f811e8e None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:50.455407 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a939b-e82e-4e09-8a2f-da312f811e8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:50.456803 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4806999a-4050-4eea-9852-1adfc1519948 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:50.462643 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 470/932] 10.222.0.13 () {56 vars in 1112 bytes} [Tue Jun 7 21:50:50 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:50.471500 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4806999a-4050-4eea-9852-1adfc1519948 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:50.471671 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2a939b-e82e-4e09-8a2f-da312f811e8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:50.471671 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2a939b-e82e-4e09-8a2f-da312f811e8e None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:50.480860 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:50.492460 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-1f2a939b-e82e-4e09-8a2f-da312f811e8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:50.492460 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:50.492460 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:50.492460 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:50.492460 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:50.492460 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:50.492460 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:50.492460 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:50.492460 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:50.492460 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:50.492460 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:50.492460 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:50.492460 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:50.492460 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:50.492460 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:50.493426 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:50.493426 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:50.493426 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:50.493426 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:50.493426 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:50.493426 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:50.493426 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:50.493426 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:50.493426 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:50.493426 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:50.493426 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:50.493426 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:50.493426 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:50.493426 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:50.493426 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:50.493426 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:50.493426 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:50.494341 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:50.494341 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:50.494341 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:50.494341 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:50.501207 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 463/933] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:50:50.502863 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a939b-e82e-4e09-8a2f-da312f811e8e None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:50.585741 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4806999a-4050-4eea-9852-1adfc1519948 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:50.617500 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a939b-e82e-4e09-8a2f-da312f811e8e None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:50.622028 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2832fc7c-e8c5-4451-90f1-d402c3a44c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:50.622693 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832fc7c-e8c5-4451-90f1-d402c3a44c60 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:50.622974 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832fc7c-e8c5-4451-90f1-d402c3a44c60 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:50.623253 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832fc7c-e8c5-4451-90f1-d402c3a44c60 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:50.623997 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2832fc7c-e8c5-4451-90f1-d402c3a44c60 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:50.624328 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2832fc7c-e8c5-4451-90f1-d402c3a44c60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:50.625687 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2832fc7c-e8c5-4451-90f1-d402c3a44c60 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:50.630855 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 471/934] 10.222.0.13 () {56 vars in 1112 bytes} [Tue Jun 7 21:50:50 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:50.633587 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4375e8c-4569-44b2-8cb0-66d6720e6c97 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:50.634091 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4375e8c-4569-44b2-8cb0-66d6720e6c97 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:50.634556 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4375e8c-4569-44b2-8cb0-66d6720e6c97 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:50.637176 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 472/935] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:50.649511 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb433a7-4fac-4619-847a-3cd7127c635a None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:50.650613 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb433a7-4fac-4619-847a-3cd7127c635a None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:50.651389 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb433a7-4fac-4619-847a-3cd7127c635a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:50.655718 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510b742-9c2a-4fab-bd72-be712f6aca55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:50.656523 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510b742-9c2a-4fab-bd72-be712f6aca55 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:50.656876 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510b742-9c2a-4fab-bd72-be712f6aca55 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:50.657234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510b742-9c2a-4fab-bd72-be712f6aca55 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:50.658631 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7510b742-9c2a-4fab-bd72-be712f6aca55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:50.659020 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7510b742-9c2a-4fab-bd72-be712f6aca55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:50.660611 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7510b742-9c2a-4fab-bd72-be712f6aca55 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:50.677351 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-8cb433a7-4fac-4619-847a-3cd7127c635a None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:50.683013 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 464/936] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:50.683244 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-8cb433a7-4fac-4619-847a-3cd7127c635a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:50.685190 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510b742-9c2a-4fab-bd72-be712f6aca55 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:50.724245 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:50.762068 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:50.787313 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 473/937] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:50:50.811059 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb433a7-4fac-4619-847a-3cd7127c635a None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:50.815277 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510b742-9c2a-4fab-bd72-be712f6aca55 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:50.862881 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af448cf1-145b-484b-a4e9-aa17a97f38cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:50.863688 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af448cf1-145b-484b-a4e9-aa17a97f38cc None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:50.863959 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af448cf1-145b-484b-a4e9-aa17a97f38cc None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:50.864239 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af448cf1-145b-484b-a4e9-aa17a97f38cc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:50.872776 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af448cf1-145b-484b-a4e9-aa17a97f38cc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:50.874310 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af448cf1-145b-484b-a4e9-aa17a97f38cc None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:50.891583 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-af448cf1-145b-484b-a4e9-aa17a97f38cc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 21:50:50.891583 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:50.891583 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:50.891583 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:50.891583 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:50.891583 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:50.891583 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:50.891583 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:50.891583 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:50.891583 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:50.891583 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:50.891583 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:50.891583 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 21:50:50.891583 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 21:50:50.891583 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 21:50:50.891583 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 21:50:50.893086 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:50.893086 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:50.893086 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 21:50:50.893086 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:50.893086 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 21:50:50.893086 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:50.893086 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:50.893086 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:50.893086 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:50.893086 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:50.893086 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:50.893086 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:50.893086 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:50.893086 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:50.893086 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:50.893086 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:50.893086 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:50.893086 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:50.894046 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:50.894046 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:50.894046 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 21:50:50.894046 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 21:50:50.894046 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 21:50:50.894046 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 21:50:50.894046 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 21:50:50.894046 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 21:50:50.894046 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 21:50:50.894046 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:50.894046 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 465/938] 10.222.0.13 () {56 vars in 1117 bytes} [Tue Jun 7 21:50:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:50:50.899497 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af448cf1-145b-484b-a4e9-aa17a97f38cc None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:50.930729 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb433a7-4fac-4619-847a-3cd7127c635a None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:50.974654 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d6427e-5b12-4ccb-9b4a-85b3ca296314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:50.975345 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d6427e-5b12-4ccb-9b4a-85b3ca296314 None admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:50.975638 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d6427e-5b12-4ccb-9b4a-85b3ca296314 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:50.975918 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d6427e-5b12-4ccb-9b4a-85b3ca296314 None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:50.976629 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d6427e-5b12-4ccb-9b4a-85b3ca296314 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:50.978065 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d6427e-5b12-4ccb-9b4a-85b3ca296314 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:50.979458 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: WARNING keystone.server.flask.common [None req-80d6427e-5b12-4ccb-9b4a-85b3ca296314 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 07 21:50:51.000110 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.sql.core [None req-80d6427e-5b12-4ccb-9b4a-85b3ca296314 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 07 21:50:51.000110 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 07 21:50:51.000110 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [parameters: {'id': '1a77591aa09b4d71a39db44f8e951611', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 07 21:50:51.000110 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80446) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 07 21:50:51.018681 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-80d6427e-5b12-4ccb-9b4a-85b3ca296314 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1a77591aa09b4d71a39db44f8e951611.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1a77591aa09b4d71a39db44f8e951611. Jun 07 21:50:51.018681 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:51.018681 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 21:50:51.018681 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 21:50:51.018681 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 21:50:51.018681 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 21:50:51.018681 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 21:50:51.018681 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 21:50:51.018681 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 21:50:51.018681 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application conn.query(q) Jun 07 21:50:51.018681 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 21:50:51.018681 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 21:50:51.018681 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 21:50:51.019820 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application result.read() Jun 07 21:50:51.019820 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 21:50:51.019820 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 21:50:51.019820 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 21:50:51.019820 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 21:50:51.019820 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 21:50:51.019820 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 21:50:51.019820 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 21:50:51.019820 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 21:50:51.019820 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 07 21:50:51.019820 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:51.019820 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 07 21:50:51.019820 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:51.019820 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:51.019820 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 07 21:50:51.019820 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 07 21:50:51.019820 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 07 21:50:51.019820 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 07 21:50:51.020834 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 21:50:51.020834 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application next(self.gen) Jun 07 21:50:51.020834 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 07 21:50:51.020834 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application yield resource Jun 07 21:50:51.020834 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 21:50:51.020834 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application next(self.gen) Jun 07 21:50:51.020834 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 07 21:50:51.020834 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self.session.rollback() Jun 07 21:50:51.020834 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 07 21:50:51.020834 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self.force_reraise() Jun 07 21:50:51.020834 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 07 21:50:51.020834 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise self.value Jun 07 21:50:51.020834 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 07 21:50:51.020834 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 07 21:50:51.020834 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 07 21:50:51.020834 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application session.commit() Jun 07 21:50:51.020834 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 07 21:50:51.021752 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 07 21:50:51.021752 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 07 21:50:51.021752 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._prepare_impl() Jun 07 21:50:51.021752 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 07 21:50:51.021752 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self.session.flush() Jun 07 21:50:51.021752 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 07 21:50:51.021752 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._flush(objects) Jun 07 21:50:51.021752 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 07 21:50:51.021752 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 07 21:50:51.021752 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 07 21:50:51.021752 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application compat.raise_( Jun 07 21:50:51.021752 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 21:50:51.021752 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception Jun 07 21:50:51.021752 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 07 21:50:51.021752 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application flush_context.execute() Jun 07 21:50:51.021752 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 07 21:50:51.021752 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rec.execute(self) Jun 07 21:50:51.022831 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 07 21:50:51.022831 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 07 21:50:51.022831 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 07 21:50:51.022831 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 07 21:50:51.022831 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 07 21:50:51.022831 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 07 21:50:51.022831 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 07 21:50:51.022831 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 07 21:50:51.022831 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 07 21:50:51.022831 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 07 21:50:51.022831 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 07 21:50:51.022831 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 07 21:50:51.022831 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 07 21:50:51.022831 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 07 21:50:51.022831 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 07 21:50:51.024735 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 07 21:50:51.024735 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 21:50:51.024735 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception Jun 07 21:50:51.024735 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 21:50:51.024735 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 21:50:51.024735 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 21:50:51.024735 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 21:50:51.024735 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 21:50:51.024735 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 21:50:51.024735 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 21:50:51.024735 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application conn.query(q) Jun 07 21:50:51.024735 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 21:50:51.024735 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 21:50:51.024735 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 21:50:51.024735 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application result.read() Jun 07 21:50:51.024735 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 21:50:51.024735 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 21:50:51.026893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 21:50:51.026893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 21:50:51.026893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 21:50:51.026893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 21:50:51.026893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 21:50:51.026893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 21:50:51.026893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 07 21:50:51.026893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 07 21:50:51.026893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [parameters: {'id': '1a77591aa09b4d71a39db44f8e951611', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 07 21:50:51.026893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 07 21:50:51.026893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:51.026893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 21:50:51.026893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:51.026893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:51.026893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 07 21:50:51.028894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 07 21:50:51.028894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 07 21:50:51.028894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 07 21:50:51.028894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 07 21:50:51.028894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:51.028894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 21:50:51.028894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:51.028894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:51.028894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:51.028894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:51.028894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:51.028894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:51.028894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:51.028894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:51.028894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:51.028894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:51.028894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:51.031195 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:51.031195 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 07 21:50:51.031195 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 07 21:50:51.031195 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:51.031195 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:51.031195 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 07 21:50:51.031195 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 07 21:50:51.031195 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1a77591aa09b4d71a39db44f8e951611. Jun 07 21:50:51.031195 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:51.031195 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 474/939] 10.222.0.13 () {60 vars in 1140 bytes} [Tue Jun 7 21:50:50 2022] POST /identity/v3/projects => generated 262 bytes in 210 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:50:51.031195 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7fbb81-11fa-4c11-a19d-88396c20f7a9 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:51.031195 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7fbb81-11fa-4c11-a19d-88396c20f7a9 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:51.032923 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7fbb81-11fa-4c11-a19d-88396c20f7a9 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:51.032923 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 475/940] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:51.032923 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7fbb81-11fa-4c11-a19d-88396c20f7a9 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:51.037542 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af448cf1-145b-484b-a4e9-aa17a97f38cc None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:51.053227 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:51.061216 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:51.079826 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bd476e-65df-4b7b-ae53-31050c7e7c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:51.080889 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd476e-65df-4b7b-ae53-31050c7e7c62 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:51.081178 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd476e-65df-4b7b-ae53-31050c7e7c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:51.081461 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd476e-65df-4b7b-ae53-31050c7e7c62 None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:51.082471 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bd476e-65df-4b7b-ae53-31050c7e7c62 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:51.083169 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bd476e-65df-4b7b-ae53-31050c7e7c62 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:51.084915 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bd476e-65df-4b7b-ae53-31050c7e7c62 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:51.103249 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 466/941] 10.222.0.13 () {56 vars in 1149 bytes} [Tue Jun 7 21:50:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:51.103249 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab0b50-7e46-43e4-b596-0f3587a420cf None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:51.103249 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab0b50-7e46-43e4-b596-0f3587a420cf None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:51.103249 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab0b50-7e46-43e4-b596-0f3587a420cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:51.125499 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-6fab0b50-7e46-43e4-b596-0f3587a420cf None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:51.126742 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-6fab0b50-7e46-43e4-b596-0f3587a420cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:51.145032 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7fbb81-11fa-4c11-a19d-88396c20f7a9 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:51.185038 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c9ece-fbb5-49eb-ba7e-a14fd25ea1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:51.186374 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c9ece-fbb5-49eb-ba7e-a14fd25ea1bb None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:51.186653 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c9ece-fbb5-49eb-ba7e-a14fd25ea1bb None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:51.187080 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c9ece-fbb5-49eb-ba7e-a14fd25ea1bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:51.194277 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5c9ece-fbb5-49eb-ba7e-a14fd25ea1bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:51.195877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5c9ece-fbb5-49eb-ba7e-a14fd25ea1bb None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:51.202009 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 467/942] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:50:51.203238 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fab0b50-7e46-43e4-b596-0f3587a420cf None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:51.215854 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-7c5c9ece-fbb5-49eb-ba7e-a14fd25ea1bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:51.215854 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:51.215854 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:51.215854 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:51.215854 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:51.215854 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:51.215854 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:51.215854 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:51.215854 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:51.215854 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:51.215854 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:51.215854 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:51.215854 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 21:50:51.215854 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 21:50:51.215854 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 21:50:51.217430 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 21:50:51.217430 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:51.217430 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:51.217430 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:51.217430 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:51.217430 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:51.217430 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:51.217430 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:51.217430 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:51.217430 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:51.217430 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:51.217430 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:51.217430 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:51.217430 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:51.217430 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:51.217430 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:51.217430 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:51.219700 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 21:50:51.219700 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 21:50:51.219700 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 21:50:51.219700 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 21:50:51.219700 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 21:50:51.219700 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 21:50:51.219700 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:51.219700 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:51.222864 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 476/943] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:50:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:51.223754 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c9ece-fbb5-49eb-ba7e-a14fd25ea1bb None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:51.326026 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:51.355490 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fab0b50-7e46-43e4-b596-0f3587a420cf None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:51.359251 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c9ece-fbb5-49eb-ba7e-a14fd25ea1bb None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:51.365964 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:51.393141 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f693f7-4f27-4c84-9cd2-0dfb888f06ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:51.393802 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f693f7-4f27-4c84-9cd2-0dfb888f06ab None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:51.394081 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f693f7-4f27-4c84-9cd2-0dfb888f06ab None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:51.394374 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f693f7-4f27-4c84-9cd2-0dfb888f06ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:51.398311 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d3a867-ac98-4bb2-9130-38e6cfd4bdbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:51.399191 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3a867-ac98-4bb2-9130-38e6cfd4bdbc None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:51.399671 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3a867-ac98-4bb2-9130-38e6cfd4bdbc None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:51.400122 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3a867-ac98-4bb2-9130-38e6cfd4bdbc None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:51.401061 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d3a867-ac98-4bb2-9130-38e6cfd4bdbc None admin] RBAC: Adding query filter params (name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:51.401356 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d3a867-ac98-4bb2-9130-38e6cfd4bdbc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:51.402791 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d3a867-ac98-4bb2-9130-38e6cfd4bdbc None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:51.404981 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f693f7-4f27-4c84-9cd2-0dfb888f06ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:51.406415 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f693f7-4f27-4c84-9cd2-0dfb888f06ab None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:51.423979 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 477/944] 10.222.0.13 () {56 vars in 1112 bytes} [Tue Jun 7 21:50:51 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:51.425598 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-d3f693f7-4f27-4c84-9cd2-0dfb888f06ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:51.425598 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:51.425598 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:51.425598 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:51.425598 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:51.425598 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:51.425598 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:51.425598 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:51.425598 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:51.425598 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:51.425598 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:51.425598 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:51.425598 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:51.425598 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:51.425598 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:51.426467 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:51.426467 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:51.426467 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:51.426467 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:51.426467 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:51.426467 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:51.426467 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:51.426467 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:51.426467 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:51.426467 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:51.426467 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:51.426467 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:51.426467 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:51.426467 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:51.426467 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:51.426467 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:51.426467 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:51.427419 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:51.427419 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:51.427419 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:51.427419 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:51.428503 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d3a867-ac98-4bb2-9130-38e6cfd4bdbc None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:51.431747 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 468/945] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:51.432810 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f693f7-4f27-4c84-9cd2-0dfb888f06ab None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:51.569234 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d3a867-ac98-4bb2-9130-38e6cfd4bdbc None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:51.571956 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f693f7-4f27-4c84-9cd2-0dfb888f06ab None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:51.610165 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32fac08-c34d-4e7c-b8f8-e2e2f439a1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:51.611818 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32fac08-c34d-4e7c-b8f8-e2e2f439a1d9 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:51.612415 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32fac08-c34d-4e7c-b8f8-e2e2f439a1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:51.612892 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32fac08-c34d-4e7c-b8f8-e2e2f439a1d9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:51.613807 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32fac08-c34d-4e7c-b8f8-e2e2f439a1d9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:51.615448 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32fac08-c34d-4e7c-b8f8-e2e2f439a1d9 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:51.621201 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c43fc-ee4a-4301-adf9-a8348b52243e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:51.621875 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c43fc-ee4a-4301-adf9-a8348b52243e None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:51.622169 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c43fc-ee4a-4301-adf9-a8348b52243e None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:51.622447 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c43fc-ee4a-4301-adf9-a8348b52243e None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:51.623184 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377c43fc-ee4a-4301-adf9-a8348b52243e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:51.623522 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377c43fc-ee4a-4301-adf9-a8348b52243e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:51.624909 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377c43fc-ee4a-4301-adf9-a8348b52243e None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:51.631455 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-e32fac08-c34d-4e7c-b8f8-e2e2f439a1d9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 21:50:51.631455 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:51.631455 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 21:50:51.631455 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 21:50:51.631455 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 21:50:51.631455 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 21:50:51.631455 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 21:50:51.631455 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:50:51.631455 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 21:50:51.631455 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:51.631455 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 21:50:51.631455 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:51.631455 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:51.631455 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:51.631455 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:51.631455 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:51.631455 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:51.632711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:51.632711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:51.632711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:51.632711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:51.632711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:51.632711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:51.632711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 21:50:51.632711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 21:50:51.632711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 21:50:51.632711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 21:50:51.632711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:51.632711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:51.632711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:51.632711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:51.632711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:51.632711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:51.633782 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:51.633782 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:51.633782 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:51.633782 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:51.633782 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:51.633782 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:51.633782 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:51.633782 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:51.633782 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:51.633782 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:51.633782 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 21:50:51.633782 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:50:51.633782 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 21:50:51.633782 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:51.636408 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 478/946] 10.222.0.13 () {56 vars in 1126 bytes} [Tue Jun 7 21:50:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:51.637074 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 469/947] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:51.643593 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c43fc-ee4a-4301-adf9-a8348b52243e None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:51.643818 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32fac08-c34d-4e7c-b8f8-e2e2f439a1d9 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:51.648191 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:51.668792 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:51.771353 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c43fc-ee4a-4301-adf9-a8348b52243e None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:51.772117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32fac08-c34d-4e7c-b8f8-e2e2f439a1d9 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:51.811160 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e163591-e751-47b6-a0c0-33468f55940f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:51.811906 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509bdaf4-f8dd-4943-a3fc-8fa6da417298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:51.812367 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e163591-e751-47b6-a0c0-33468f55940f None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:51.812367 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e163591-e751-47b6-a0c0-33468f55940f None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:51.812631 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e163591-e751-47b6-a0c0-33468f55940f None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:51.812729 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509bdaf4-f8dd-4943-a3fc-8fa6da417298 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:51.812980 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509bdaf4-f8dd-4943-a3fc-8fa6da417298 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:51.813258 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509bdaf4-f8dd-4943-a3fc-8fa6da417298 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:51.813360 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e163591-e751-47b6-a0c0-33468f55940f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:51.813698 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e163591-e751-47b6-a0c0-33468f55940f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:51.815665 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e163591-e751-47b6-a0c0-33468f55940f None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:51.820621 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 470/948] 10.222.0.13 () {56 vars in 1122 bytes} [Tue Jun 7 21:50:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:51.821984 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509bdaf4-f8dd-4943-a3fc-8fa6da417298 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:51.823480 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509bdaf4-f8dd-4943-a3fc-8fa6da417298 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:51.826410 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e163591-e751-47b6-a0c0-33468f55940f None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:51.843897 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-509bdaf4-f8dd-4943-a3fc-8fa6da417298 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:51.843897 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:51.843897 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:51.843897 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:51.843897 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:51.843897 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:51.843897 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:51.843897 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:51.843897 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:51.843897 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:51.843897 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:51.843897 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:51.843897 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 21:50:51.843897 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 21:50:51.843897 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 21:50:51.845723 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 21:50:51.845723 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:51.845723 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:51.845723 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:51.845723 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:51.845723 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:51.845723 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:51.845723 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:51.845723 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:51.845723 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:51.845723 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:51.845723 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:51.845723 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:51.845723 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:51.845723 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:51.845723 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:51.845723 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:51.846825 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 21:50:51.846825 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 21:50:51.846825 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 21:50:51.846825 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 21:50:51.846825 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 21:50:51.846825 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 21:50:51.846825 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:51.846825 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:51.846825 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 479/949] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:50:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:51.852332 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509bdaf4-f8dd-4943-a3fc-8fa6da417298 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:51.930422 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:51.935403 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:51.956231 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e163591-e751-47b6-a0c0-33468f55940f None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:51.979545 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509bdaf4-f8dd-4943-a3fc-8fa6da417298 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:52.001635 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aac13b-ae93-4baf-8ff4-ab2aa40ebc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:52.003319 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aac13b-ae93-4baf-8ff4-ab2aa40ebc95 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:52.003839 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aac13b-ae93-4baf-8ff4-ab2aa40ebc95 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:52.004570 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aac13b-ae93-4baf-8ff4-ab2aa40ebc95 None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:52.005757 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aac13b-ae93-4baf-8ff4-ab2aa40ebc95 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:52.006685 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aac13b-ae93-4baf-8ff4-ab2aa40ebc95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:52.008827 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aac13b-ae93-4baf-8ff4-ab2aa40ebc95 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:52.021342 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434ecfad-201e-4283-b1ee-9e03f73c0246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:52.021342 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ecfad-201e-4283-b1ee-9e03f73c0246 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:52.021342 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ecfad-201e-4283-b1ee-9e03f73c0246 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:52.021342 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ecfad-201e-4283-b1ee-9e03f73c0246 None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:52.021733 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434ecfad-201e-4283-b1ee-9e03f73c0246 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:52.021733 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434ecfad-201e-4283-b1ee-9e03f73c0246 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:52.022119 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 471/950] 10.222.0.13 () {56 vars in 1122 bytes} [Tue Jun 7 21:50:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:52.022644 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434ecfad-201e-4283-b1ee-9e03f73c0246 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:52.023764 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aac13b-ae93-4baf-8ff4-ab2aa40ebc95 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:52.036345 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 480/951] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jun 7 21:50:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:50:52.043660 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434ecfad-201e-4283-b1ee-9e03f73c0246 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:52.132410 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aac13b-ae93-4baf-8ff4-ab2aa40ebc95 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:52.149404 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434ecfad-201e-4283-b1ee-9e03f73c0246 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:52.168649 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2ed7f4-928d-4486-80a5-f8bcb1373284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:52.169698 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2ed7f4-928d-4486-80a5-f8bcb1373284 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:52.170180 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2ed7f4-928d-4486-80a5-f8bcb1373284 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:52.170656 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2ed7f4-928d-4486-80a5-f8bcb1373284 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:52.171674 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2ed7f4-928d-4486-80a5-f8bcb1373284 None admin] RBAC: Adding query filter params (interface=public, service_id=b4ca70b9a07542b4a05ba279e5ff0fe5, region_id=RegionOne) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:52.172204 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2ed7f4-928d-4486-80a5-f8bcb1373284 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:52.173923 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2ed7f4-928d-4486-80a5-f8bcb1373284 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:52.174282 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:52.179976 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 472/952] 10.222.0.13 () {56 vars in 1251 bytes} [Tue Jun 7 21:50:52 2022] GET /identity/v3/endpoints?service_id=b4ca70b9a07542b4a05ba279e5ff0fe5&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:52.185920 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2ed7f4-928d-4486-80a5-f8bcb1373284 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:52.192740 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598fa2f6-a7de-465b-b2fa-ae7a8c2f3c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:52.193902 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598fa2f6-a7de-465b-b2fa-ae7a8c2f3c92 None admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:52.194381 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598fa2f6-a7de-465b-b2fa-ae7a8c2f3c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:52.194853 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598fa2f6-a7de-465b-b2fa-ae7a8c2f3c92 None admin] PATH_INFO: `/v3/projects/0219dbda54274fdc9bdccbd960c30c6b/users/86619a80df8841a3ba7d8471ac38056a/roles/14b78e4dcfb84633981a9b2f2f06f4fb` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:52.197995 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598fa2f6-a7de-465b-b2fa-ae7a8c2f3c92 None admin] RBAC: Authorizing `identity:create_grant(project_id=0219dbda54274fdc9bdccbd960c30c6b, user_id=86619a80df8841a3ba7d8471ac38056a, role_id=14b78e4dcfb84633981a9b2f2f06f4fb)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:52.200516 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598fa2f6-a7de-465b-b2fa-ae7a8c2f3c92 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:52.202781 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:52.227571 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 481/953] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jun 7 21:50:52 2022] PUT /identity/v3/projects/0219dbda54274fdc9bdccbd960c30c6b/users/86619a80df8841a3ba7d8471ac38056a/roles/14b78e4dcfb84633981a9b2f2f06f4fb => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 21:50:52.309459 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2ed7f4-928d-4486-80a5-f8bcb1373284 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:52.348708 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d50ce20-fa0a-4886-ac27-dbe5781be56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:52.349405 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d50ce20-fa0a-4886-ac27-dbe5781be56f None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:52.349689 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d50ce20-fa0a-4886-ac27-dbe5781be56f None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:52.349969 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d50ce20-fa0a-4886-ac27-dbe5781be56f None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:52.350664 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d50ce20-fa0a-4886-ac27-dbe5781be56f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:52.352191 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d50ce20-fa0a-4886-ac27-dbe5781be56f None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:52.363523 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 473/954] 10.222.0.13 () {56 vars in 1087 bytes} [Tue Jun 7 21:50:52 2022] GET /identity/v3/services => generated 1497 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 21:50:53.046157 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f082b7f-a0f5-472c-a263-52940b88a51b None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:53.046670 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f082b7f-a0f5-472c-a263-52940b88a51b None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:53.046748 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f082b7f-a0f5-472c-a263-52940b88a51b None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:53.048423 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 482/955] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:53.053827 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db6112-4f35-4cac-a726-df5ede4c7f27 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:53.054253 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db6112-4f35-4cac-a726-df5ede4c7f27 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:53.054432 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db6112-4f35-4cac-a726-df5ede4c7f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:53.071983 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-31db6112-4f35-4cac-a726-df5ede4c7f27 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:53.073166 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-31db6112-4f35-4cac-a726-df5ede4c7f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:53.142922 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 474/956] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 21:50:53.165194 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f082b7f-a0f5-472c-a263-52940b88a51b None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:53.273111 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f082b7f-a0f5-472c-a263-52940b88a51b None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:53.311126 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31430afd-fc83-4542-ad41-a475f64a2feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:53.311748 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31430afd-fc83-4542-ad41-a475f64a2feb None admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:53.312297 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31430afd-fc83-4542-ad41-a475f64a2feb None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:53.312574 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31430afd-fc83-4542-ad41-a475f64a2feb None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:53.313363 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31430afd-fc83-4542-ad41-a475f64a2feb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:53.314915 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31430afd-fc83-4542-ad41-a475f64a2feb None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:53.339714 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 483/957] 10.222.0.13 () {60 vars in 1141 bytes} [Tue Jun 7 21:50:53 2022] POST /identity/v3/services => generated 242 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 21:50:53.855719 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a97bd-e879-48de-981b-91a15d4d520c None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:53.856046 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a97bd-e879-48de-981b-91a15d4d520c None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:53.856275 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a97bd-e879-48de-981b-91a15d4d520c None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:53.857892 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 475/958] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:53.863909 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f06a02-8e62-429c-bb5a-206e83e16551 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:53.864205 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f06a02-8e62-429c-bb5a-206e83e16551 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:53.864488 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f06a02-8e62-429c-bb5a-206e83e16551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:53.882089 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-44f06a02-8e62-429c-bb5a-206e83e16551 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:53.883624 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-44f06a02-8e62-429c-bb5a-206e83e16551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:53.926110 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:53.961846 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 484/959] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:53 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:50:53.991889 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336a97bd-e879-48de-981b-91a15d4d520c None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:54.001097 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:54.099646 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336a97bd-e879-48de-981b-91a15d4d520c None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:54.135970 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6bc7a5-25ea-41e3-8b13-8e8d165be861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:54.136646 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6bc7a5-25ea-41e3-8b13-8e8d165be861 None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:54.136930 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6bc7a5-25ea-41e3-8b13-8e8d165be861 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:54.137209 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6bc7a5-25ea-41e3-8b13-8e8d165be861 None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:54.137922 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6bc7a5-25ea-41e3-8b13-8e8d165be861 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:54.139363 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6bc7a5-25ea-41e3-8b13-8e8d165be861 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:54.140685 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: WARNING keystone.server.flask.common [None req-1b6bc7a5-25ea-41e3-8b13-8e8d165be861 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 07 21:50:54.155165 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.sql.core [None req-1b6bc7a5-25ea-41e3-8b13-8e8d165be861 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 07 21:50:54.155165 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 07 21:50:54.155165 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [parameters: {'user_id': '70515402bd3b4e0ca68a39a6f26c1e51', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 07 21:50:54.155165 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80445) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 07 21:50:54.163017 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-1b6bc7a5-25ea-41e3-8b13-8e8d165be861 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 07 21:50:54.163017 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:54.163017 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 21:50:54.163017 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 21:50:54.163017 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 21:50:54.163017 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 21:50:54.163017 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 21:50:54.163017 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 21:50:54.163017 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 21:50:54.163017 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application conn.query(q) Jun 07 21:50:54.163017 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 21:50:54.163017 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 21:50:54.163017 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 21:50:54.163017 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application result.read() Jun 07 21:50:54.163017 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 21:50:54.163954 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 21:50:54.163954 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 21:50:54.163954 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 21:50:54.163954 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 21:50:54.163954 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 21:50:54.163954 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 21:50:54.163954 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 21:50:54.163954 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 07 21:50:54.163954 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:54.163954 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 07 21:50:54.163954 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:54.163954 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:54.163954 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 07 21:50:54.163954 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 07 21:50:54.163954 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 07 21:50:54.163954 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 07 21:50:54.163954 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 21:50:54.163954 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application next(self.gen) Jun 07 21:50:54.164921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 07 21:50:54.164921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application yield resource Jun 07 21:50:54.164921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 21:50:54.164921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application next(self.gen) Jun 07 21:50:54.164921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 07 21:50:54.164921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self.session.rollback() Jun 07 21:50:54.164921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 07 21:50:54.164921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self.force_reraise() Jun 07 21:50:54.164921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 07 21:50:54.164921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise self.value Jun 07 21:50:54.164921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 07 21:50:54.164921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 07 21:50:54.164921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 07 21:50:54.164921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application session.commit() Jun 07 21:50:54.164921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 07 21:50:54.164921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 07 21:50:54.164921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 07 21:50:54.165832 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._prepare_impl() Jun 07 21:50:54.165832 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 07 21:50:54.165832 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self.session.flush() Jun 07 21:50:54.165832 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 07 21:50:54.165832 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._flush(objects) Jun 07 21:50:54.165832 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 07 21:50:54.165832 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 07 21:50:54.165832 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 07 21:50:54.165832 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application compat.raise_( Jun 07 21:50:54.165832 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 21:50:54.165832 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception Jun 07 21:50:54.165832 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 07 21:50:54.165832 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application flush_context.execute() Jun 07 21:50:54.165832 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 07 21:50:54.165832 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rec.execute(self) Jun 07 21:50:54.165832 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 07 21:50:54.165832 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 07 21:50:54.166727 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 07 21:50:54.166727 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 07 21:50:54.166727 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 07 21:50:54.166727 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 07 21:50:54.166727 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 07 21:50:54.166727 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 07 21:50:54.166727 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 07 21:50:54.166727 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 07 21:50:54.166727 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 07 21:50:54.166727 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 07 21:50:54.166727 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 07 21:50:54.166727 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 07 21:50:54.166727 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 07 21:50:54.166727 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 07 21:50:54.166727 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 21:50:54.166727 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception Jun 07 21:50:54.167626 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 21:50:54.167626 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 21:50:54.167626 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 21:50:54.167626 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 21:50:54.167626 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 21:50:54.167626 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 21:50:54.167626 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 21:50:54.167626 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application conn.query(q) Jun 07 21:50:54.167626 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 21:50:54.167626 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 21:50:54.167626 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 21:50:54.167626 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application result.read() Jun 07 21:50:54.167626 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 21:50:54.167626 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 21:50:54.167626 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 21:50:54.167626 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 21:50:54.168453 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 21:50:54.168453 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 21:50:54.168453 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 21:50:54.168453 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 21:50:54.168453 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 07 21:50:54.168453 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 07 21:50:54.168453 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [parameters: {'user_id': '70515402bd3b4e0ca68a39a6f26c1e51', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 07 21:50:54.168453 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 07 21:50:54.168453 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:54.168453 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 21:50:54.168453 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:54.168453 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:54.168453 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:54.168453 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:54.168453 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:54.169254 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:54.169254 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:54.169254 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:54.169254 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:54.169254 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:54.169254 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:54.169254 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:54.169254 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 07 21:50:54.169254 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 07 21:50:54.169254 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:54.169254 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:54.169254 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 21:50:54.169254 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:54.169254 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 21:50:54.169254 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:54.169254 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 07 21:50:54.169254 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 07 21:50:54.170200 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 07 21:50:54.170200 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 07 21:50:54.170200 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 07 21:50:54.170200 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 07 21:50:54.170200 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 07 21:50:54.170200 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:54.170200 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 476/960] 10.222.0.13 () {60 vars in 1131 bytes} [Tue Jun 7 21:50:53 2022] POST /identity/v3/users => generated 160 bytes in 173 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:54.170596 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f06a02-8e62-429c-bb5a-206e83e16551 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:54.276541 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f06a02-8e62-429c-bb5a-206e83e16551 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:54.316161 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ef2554-1976-422f-8afb-f457febba6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:54.316991 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef2554-1976-422f-8afb-f457febba6bf None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:54.317272 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef2554-1976-422f-8afb-f457febba6bf None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:54.317544 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef2554-1976-422f-8afb-f457febba6bf None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:54.326111 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ef2554-1976-422f-8afb-f457febba6bf None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:54.327608 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ef2554-1976-422f-8afb-f457febba6bf None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:54.343128 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-f9ef2554-1976-422f-8afb-f457febba6bf None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 21:50:54.343128 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:54.343128 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:54.343128 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:54.343128 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:54.343128 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:54.343128 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:54.343128 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:54.343128 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:54.343128 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:54.343128 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:54.343128 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:54.343128 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 21:50:54.343128 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 21:50:54.343128 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 21:50:54.343128 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 21:50:54.344141 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:54.344141 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:54.344141 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 21:50:54.344141 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:54.344141 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 21:50:54.344141 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:54.344141 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:54.344141 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:54.344141 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:54.344141 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:54.344141 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:54.344141 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:54.344141 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:54.344141 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:54.344141 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:54.344141 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:54.344141 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:54.344141 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:54.346139 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:54.346139 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:54.346139 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 21:50:54.346139 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 21:50:54.346139 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 21:50:54.346139 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 21:50:54.346139 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 21:50:54.346139 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 21:50:54.346139 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 21:50:54.346139 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:54.346139 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 485/961] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:50:54 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:54.351469 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6bc7a5-25ea-41e3-8b13-8e8d165be861 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:54.430750 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:54.466571 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6bc7a5-25ea-41e3-8b13-8e8d165be861 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:54.503853 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959de8ee-1333-4b6a-ac74-40a6987df7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:54.504585 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959de8ee-1333-4b6a-ac74-40a6987df7a1 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:54.504870 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959de8ee-1333-4b6a-ac74-40a6987df7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:54.505149 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959de8ee-1333-4b6a-ac74-40a6987df7a1 None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:54.505959 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959de8ee-1333-4b6a-ac74-40a6987df7a1 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:54.506264 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959de8ee-1333-4b6a-ac74-40a6987df7a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:54.507919 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959de8ee-1333-4b6a-ac74-40a6987df7a1 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:54.548836 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 477/962] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jun 7 21:50:54 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:54.660989 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c57a4e-5f91-4a2e-bde2-43d6be84ae8a None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:54.661619 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c57a4e-5f91-4a2e-bde2-43d6be84ae8a None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:54.662106 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c57a4e-5f91-4a2e-bde2-43d6be84ae8a None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:54.664012 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 486/963] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:54.670077 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aabe72-77b9-45e0-aad9-978986b79cb2 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:54.670688 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aabe72-77b9-45e0-aad9-978986b79cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:54.671201 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aabe72-77b9-45e0-aad9-978986b79cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:54.690358 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-57aabe72-77b9-45e0-aad9-978986b79cb2 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:54.692341 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-57aabe72-77b9-45e0-aad9-978986b79cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:54.765319 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea996133-c1af-4b99-b718-fc6e0bdbd4a0 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:54.765857 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea996133-c1af-4b99-b718-fc6e0bdbd4a0 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:54.766305 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea996133-c1af-4b99-b718-fc6e0bdbd4a0 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:54.768378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 487/964] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 21:50:54.776095 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8441bbc6-df68-4682-b83e-dbfc0d29b10b None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:54.776600 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8441bbc6-df68-4682-b83e-dbfc0d29b10b None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:54.777080 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8441bbc6-df68-4682-b83e-dbfc0d29b10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:54.779031 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 478/965] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:54 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:50:54.797490 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-8441bbc6-df68-4682-b83e-dbfc0d29b10b None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:54.799448 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-8441bbc6-df68-4682-b83e-dbfc0d29b10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:54.808508 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aabe72-77b9-45e0-aad9-978986b79cb2 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:54.829757 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:54.895563 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:54.896889 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 488/966] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:54 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:50:54.921248 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8441bbc6-df68-4682-b83e-dbfc0d29b10b None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:54.942537 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aabe72-77b9-45e0-aad9-978986b79cb2 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:54.983622 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a49b558-08a2-475a-983d-4a43a573aa55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:54.984444 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a49b558-08a2-475a-983d-4a43a573aa55 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:54.984731 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a49b558-08a2-475a-983d-4a43a573aa55 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:54.985014 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a49b558-08a2-475a-983d-4a43a573aa55 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:54.996672 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a49b558-08a2-475a-983d-4a43a573aa55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:50:55.005452 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-3a49b558-08a2-475a-983d-4a43a573aa55 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 21:50:55.005452 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:55.005452 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:55.005452 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:55.005452 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:55.005452 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:55.005452 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:55.005452 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:55.005452 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:55.005452 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:55.005452 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:55.005452 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:55.005452 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:50:55.005452 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:50:55.005452 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:50:55.005452 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:50:55.006395 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:50:55.006395 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:50:55.006395 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:50:55.006395 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:50:55.006395 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:55.006395 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:55.006395 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:55.006395 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:55.006395 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:55.006395 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:55.006395 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:55.006395 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:55.006395 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:55.006395 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:55.006395 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:55.006395 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:55.008632 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:55.008632 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:55.008632 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:55.008632 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:55.008632 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:50:55.008632 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:50:55.008632 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:50:55.008632 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:50:55.008632 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:50:55.008632 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:50:55.008632 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 21:50:55.008632 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:55.008632 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 479/967] 10.222.0.13 () {56 vars in 1120 bytes} [Tue Jun 7 21:50:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:50:55.012810 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a49b558-08a2-475a-983d-4a43a573aa55 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:55.038205 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8441bbc6-df68-4682-b83e-dbfc0d29b10b None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:55.073735 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d9a6ec-b0db-46dd-9585-dfbd73db209f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:55.074844 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d9a6ec-b0db-46dd-9585-dfbd73db209f None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:55.075132 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d9a6ec-b0db-46dd-9585-dfbd73db209f None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:55.075589 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d9a6ec-b0db-46dd-9585-dfbd73db209f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:55.076573 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d9a6ec-b0db-46dd-9585-dfbd73db209f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:55.078406 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d9a6ec-b0db-46dd-9585-dfbd73db209f None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:55.094486 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-69d9a6ec-b0db-46dd-9585-dfbd73db209f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 21:50:55.094486 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:55.094486 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 21:50:55.094486 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 21:50:55.094486 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 21:50:55.094486 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 21:50:55.094486 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 21:50:55.094486 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:50:55.094486 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 21:50:55.094486 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:55.094486 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 21:50:55.094486 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:55.094486 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:55.094486 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:55.094486 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:55.094486 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:55.094486 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:55.102017 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:55.102017 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:55.102017 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:55.102017 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:55.102017 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:55.102017 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:55.102017 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 21:50:55.102017 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 21:50:55.102017 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 21:50:55.102017 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 21:50:55.102017 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:55.102017 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:55.102017 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:55.102017 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:55.102017 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:55.102017 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:55.102893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:55.102893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:55.102893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:55.102893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:55.102893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:55.102893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:55.102893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:55.102893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:55.102893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:55.102893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:55.102893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 21:50:55.102893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:50:55.102893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 21:50:55.102893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:55.102893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 489/968] 10.222.0.13 () {56 vars in 1126 bytes} [Tue Jun 7 21:50:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:50:55.103720 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d9a6ec-b0db-46dd-9585-dfbd73db209f None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:55.124840 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a49b558-08a2-475a-983d-4a43a573aa55 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:55.130322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:55.145661 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:55.166874 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fd66ac-73db-414a-860e-6a65470aaeeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:55.167575 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd66ac-73db-414a-860e-6a65470aaeeb None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:55.167861 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd66ac-73db-414a-860e-6a65470aaeeb None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:55.168136 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd66ac-73db-414a-860e-6a65470aaeeb None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:55.168875 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fd66ac-73db-414a-860e-6a65470aaeeb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:55.169179 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fd66ac-73db-414a-860e-6a65470aaeeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:55.170545 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fd66ac-73db-414a-860e-6a65470aaeeb None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:55.184159 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 480/969] 10.222.0.13 () {56 vars in 1115 bytes} [Tue Jun 7 21:50:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:55.185169 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fd66ac-73db-414a-860e-6a65470aaeeb None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:55.211667 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d9a6ec-b0db-46dd-9585-dfbd73db209f None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:55.250633 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5e08a8-688c-4776-91f4-487bbd7b5404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:55.251897 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e08a8-688c-4776-91f4-487bbd7b5404 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:55.252181 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e08a8-688c-4776-91f4-487bbd7b5404 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:55.252450 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e08a8-688c-4776-91f4-487bbd7b5404 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:55.253157 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5e08a8-688c-4776-91f4-487bbd7b5404 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:55.253458 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5e08a8-688c-4776-91f4-487bbd7b5404 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:55.254822 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5e08a8-688c-4776-91f4-487bbd7b5404 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:55.259725 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 490/970] 10.222.0.13 () {56 vars in 1122 bytes} [Tue Jun 7 21:50:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:50:55.265226 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5e08a8-688c-4776-91f4-487bbd7b5404 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:55.302448 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fd66ac-73db-414a-860e-6a65470aaeeb None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:55.340877 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa119f32-e87f-4259-8385-7c3e767daab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:55.342030 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa119f32-e87f-4259-8385-7c3e767daab2 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:55.342311 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa119f32-e87f-4259-8385-7c3e767daab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:55.342584 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa119f32-e87f-4259-8385-7c3e767daab2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:55.355098 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa119f32-e87f-4259-8385-7c3e767daab2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:55.356571 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa119f32-e87f-4259-8385-7c3e767daab2 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:55.360122 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:55.375042 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5e08a8-688c-4776-91f4-487bbd7b5404 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:55.377559 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-aa119f32-e87f-4259-8385-7c3e767daab2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:55.377559 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:55.377559 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:55.377559 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:55.377559 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:55.377559 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:55.377559 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:55.377559 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:55.377559 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:55.377559 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:55.377559 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:55.377559 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:55.377559 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:55.377559 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:55.377559 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:55.378538 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:55.378538 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:55.378538 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:55.378538 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:55.378538 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:55.378538 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:55.378538 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:55.378538 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:55.378538 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:55.378538 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:55.378538 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:55.378538 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:55.378538 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:55.378538 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:55.378538 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:55.378538 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:55.378538 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:55.380187 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:55.380187 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:55.380187 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:55.380187 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:55.380187 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 481/971] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:55.389067 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa119f32-e87f-4259-8385-7c3e767daab2 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:55.419097 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc29d1c-ee13-42a2-aae4-77d4d3129114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:55.419872 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc29d1c-ee13-42a2-aae4-77d4d3129114 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:55.420270 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc29d1c-ee13-42a2-aae4-77d4d3129114 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:55.420540 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc29d1c-ee13-42a2-aae4-77d4d3129114 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:55.421241 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc29d1c-ee13-42a2-aae4-77d4d3129114 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:55.421538 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc29d1c-ee13-42a2-aae4-77d4d3129114 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:55.422769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:55.423941 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc29d1c-ee13-42a2-aae4-77d4d3129114 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:55.429754 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 491/972] 10.222.0.13 () {56 vars in 1122 bytes} [Tue Jun 7 21:50:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:50:55.434981 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc29d1c-ee13-42a2-aae4-77d4d3129114 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:55.502930 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa119f32-e87f-4259-8385-7c3e767daab2 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:55.537327 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc29d1c-ee13-42a2-aae4-77d4d3129114 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:55.539522 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545b4667-052c-43e8-b803-2e901cc8b762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:55.540355 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b4667-052c-43e8-b803-2e901cc8b762 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:55.540791 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b4667-052c-43e8-b803-2e901cc8b762 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:55.541246 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b4667-052c-43e8-b803-2e901cc8b762 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:55.542149 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545b4667-052c-43e8-b803-2e901cc8b762 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:55.542600 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545b4667-052c-43e8-b803-2e901cc8b762 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:55.544137 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545b4667-052c-43e8-b803-2e901cc8b762 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:55.557620 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 482/973] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:55.562819 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545b4667-052c-43e8-b803-2e901cc8b762 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:55.576844 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a8040f-c1da-40ad-8a37-f4b2be153e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:55.577535 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8040f-c1da-40ad-8a37-f4b2be153e6f None admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:55.577826 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8040f-c1da-40ad-8a37-f4b2be153e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:55.578101 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8040f-c1da-40ad-8a37-f4b2be153e6f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:55.578842 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a8040f-c1da-40ad-8a37-f4b2be153e6f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:55.581091 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a8040f-c1da-40ad-8a37-f4b2be153e6f None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:55.602562 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 492/974] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Jun 7 21:50:55 2022] POST /identity/v3/endpoints => generated 349 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 21:50:55.670733 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:55.683927 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545b4667-052c-43e8-b803-2e901cc8b762 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:55.715081 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe538bdd-84b5-4655-b7ea-e09e10b7e3b6 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:55.715281 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe538bdd-84b5-4655-b7ea-e09e10b7e3b6 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:55.715590 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe538bdd-84b5-4655-b7ea-e09e10b7e3b6 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:55.717241 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 493/975] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 21:50:55.723542 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c7512-d0f8-4b6f-be48-163c44bdfe5b None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:55.723833 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c7512-d0f8-4b6f-be48-163c44bdfe5b None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:55.724113 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c7512-d0f8-4b6f-be48-163c44bdfe5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:55.730602 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bea339d-4f9f-4162-84e3-feeb1d352606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:55.731529 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea339d-4f9f-4162-84e3-feeb1d352606 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:55.731817 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea339d-4f9f-4162-84e3-feeb1d352606 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:55.732096 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea339d-4f9f-4162-84e3-feeb1d352606 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:55.740540 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bea339d-4f9f-4162-84e3-feeb1d352606 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:55.741845 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-7b7c7512-d0f8-4b6f-be48-163c44bdfe5b None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:55.742020 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bea339d-4f9f-4162-84e3-feeb1d352606 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:55.743219 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-7b7c7512-d0f8-4b6f-be48-163c44bdfe5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:55.745630 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:55.758654 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-3bea339d-4f9f-4162-84e3-feeb1d352606 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 21:50:55.758654 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:55.758654 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:55.758654 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:55.758654 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:55.758654 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:55.758654 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:55.758654 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:55.758654 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:55.758654 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:55.758654 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:55.758654 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:55.758654 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 21:50:55.758654 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 21:50:55.758654 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 21:50:55.758654 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 21:50:55.759600 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:55.759600 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:55.759600 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 21:50:55.759600 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:55.759600 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 21:50:55.759600 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:50:55.759600 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:55.759600 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:55.759600 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:55.759600 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:55.759600 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:55.759600 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:55.759600 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:55.759600 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:55.759600 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:55.759600 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:55.759600 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:55.759600 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:55.760905 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:55.760905 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:55.760905 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 21:50:55.760905 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 21:50:55.760905 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 21:50:55.760905 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 21:50:55.760905 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 21:50:55.760905 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 21:50:55.760905 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 21:50:55.760905 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:55.760905 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 483/976] 10.222.0.13 () {56 vars in 1117 bytes} [Tue Jun 7 21:50:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:50:55.765767 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bea339d-4f9f-4162-84e3-feeb1d352606 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:55.830392 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 494/977] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:55 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 21:50:55.853964 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7c7512-d0f8-4b6f-be48-163c44bdfe5b None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:55.888233 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bea339d-4f9f-4162-84e3-feeb1d352606 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:55.925385 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dc2941-3374-452e-98de-a37838d88162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:55.926534 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dc2941-3374-452e-98de-a37838d88162 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:55.926837 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dc2941-3374-452e-98de-a37838d88162 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:55.927119 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dc2941-3374-452e-98de-a37838d88162 None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:55.927999 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dc2941-3374-452e-98de-a37838d88162 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:55.928313 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dc2941-3374-452e-98de-a37838d88162 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:55.929747 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dc2941-3374-452e-98de-a37838d88162 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:55.948247 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 484/978] 10.222.0.13 () {56 vars in 1149 bytes} [Tue Jun 7 21:50:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:55.953382 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dc2941-3374-452e-98de-a37838d88162 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:55.956990 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:55.960618 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7c7512-d0f8-4b6f-be48-163c44bdfe5b None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:55.997998 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3a956-4e0b-47df-aab9-029536d471e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:55.998722 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3a956-4e0b-47df-aab9-029536d471e9 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:55.999000 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3a956-4e0b-47df-aab9-029536d471e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:55.999273 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3a956-4e0b-47df-aab9-029536d471e9 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:55.999964 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f3a956-4e0b-47df-aab9-029536d471e9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:56.001633 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f3a956-4e0b-47df-aab9-029536d471e9 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:56.014117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-a3f3a956-4e0b-47df-aab9-029536d471e9 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 21:50:56.014117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:56.014117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 21:50:56.014117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 21:50:56.014117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 21:50:56.014117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 21:50:56.014117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 21:50:56.014117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:50:56.014117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 21:50:56.014117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:56.014117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 21:50:56.014117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:56.014117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:56.014117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:56.014117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:56.014117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:56.014117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:56.015150 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:56.015150 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:56.015150 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:56.015150 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:56.015150 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:56.015150 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:56.015150 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 21:50:56.015150 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 21:50:56.015150 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 21:50:56.015150 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 21:50:56.015150 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:56.015150 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:56.015150 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:56.015150 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:56.015150 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:56.015150 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:56.016067 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:56.016067 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:56.016067 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:56.016067 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:56.016067 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:56.016067 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:56.016067 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:56.016067 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:56.016067 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:56.016067 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:56.016067 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 21:50:56.016067 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:50:56.016067 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 21:50:56.016067 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:56.016067 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 495/979] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:50:55 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:50:56.021380 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3a956-4e0b-47df-aab9-029536d471e9 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:56.024273 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:56.062435 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dc2941-3374-452e-98de-a37838d88162 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:56.099917 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50893697-3abc-415e-8097-cf4d3f9b4906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:56.100568 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50893697-3abc-415e-8097-cf4d3f9b4906 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:56.100848 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50893697-3abc-415e-8097-cf4d3f9b4906 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:56.101240 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50893697-3abc-415e-8097-cf4d3f9b4906 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:56.131015 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3a956-4e0b-47df-aab9-029536d471e9 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:56.171049 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef976ea-3f46-4955-bbac-5afc140ca76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:56.171661 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50893697-3abc-415e-8097-cf4d3f9b4906 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:56.172092 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef976ea-3f46-4955-bbac-5afc140ca76d None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:56.172092 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef976ea-3f46-4955-bbac-5afc140ca76d None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:56.172345 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef976ea-3f46-4955-bbac-5afc140ca76d None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:56.173513 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef976ea-3f46-4955-bbac-5afc140ca76d None admin] RBAC: Adding query filter params (name=compute) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:56.173679 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50893697-3abc-415e-8097-cf4d3f9b4906 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:56.173832 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef976ea-3f46-4955-bbac-5afc140ca76d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:56.175186 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef976ea-3f46-4955-bbac-5afc140ca76d None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:56.177061 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:56.181657 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 496/980] 10.222.0.13 () {56 vars in 1112 bytes} [Tue Jun 7 21:50:56 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:56.187062 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef976ea-3f46-4955-bbac-5afc140ca76d None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:56.258475 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:56.279926 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 485/981] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:50:55 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 07 21:50:56.285389 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50893697-3abc-415e-8097-cf4d3f9b4906 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:56.300158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef976ea-3f46-4955-bbac-5afc140ca76d None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:56.336325 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0fd8aa-d609-41af-bb21-abdc799534a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:56.336991 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0fd8aa-d609-41af-bb21-abdc799534a7 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:56.337277 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0fd8aa-d609-41af-bb21-abdc799534a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:56.337736 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0fd8aa-d609-41af-bb21-abdc799534a7 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:56.338254 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0fd8aa-d609-41af-bb21-abdc799534a7 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:56.338560 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0fd8aa-d609-41af-bb21-abdc799534a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:56.340019 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0fd8aa-d609-41af-bb21-abdc799534a7 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:56.345683 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 497/982] 10.222.0.13 () {56 vars in 1112 bytes} [Tue Jun 7 21:50:56 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:50:56.351406 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0fd8aa-d609-41af-bb21-abdc799534a7 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:56.391766 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50893697-3abc-415e-8097-cf4d3f9b4906 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:56.394122 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:56.428967 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c27893-f043-473c-9224-077395fa5a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:56.429762 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c27893-f043-473c-9224-077395fa5a05 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:56.430044 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c27893-f043-473c-9224-077395fa5a05 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:56.430321 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c27893-f043-473c-9224-077395fa5a05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:56.441288 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c27893-f043-473c-9224-077395fa5a05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:56.442736 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c27893-f043-473c-9224-077395fa5a05 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:56.460283 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-57c27893-f043-473c-9224-077395fa5a05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:56.460283 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:56.460283 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:56.460283 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:56.460283 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:56.460283 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:56.460283 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:56.460283 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:56.460283 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:56.460283 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:56.460283 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:56.460283 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:56.460283 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:50:56.460283 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:50:56.460283 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:50:56.461924 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:50:56.461924 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:56.461924 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:56.461924 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:56.461924 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:56.461924 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:56.461924 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:56.461924 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:56.461924 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:56.461924 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:56.461924 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:56.461924 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:56.461924 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:56.461924 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:56.461924 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:56.461924 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:56.461924 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:56.462844 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:50:56.462844 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:50:56.462844 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:50:56.462844 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:56.462844 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 486/983] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:50:56.465057 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0fd8aa-d609-41af-bb21-abdc799534a7 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:56.466846 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c27893-f043-473c-9224-077395fa5a05 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:56.496667 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:56.504914 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8349187-843e-4423-aaa7-70bae7c19be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:56.505583 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8349187-843e-4423-aaa7-70bae7c19be7 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:56.505863 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8349187-843e-4423-aaa7-70bae7c19be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:56.506140 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8349187-843e-4423-aaa7-70bae7c19be7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:56.506932 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8349187-843e-4423-aaa7-70bae7c19be7 None admin] RBAC: Adding query filter params (interface=public, service_id=5ad4118034e6440ead05b661f5708356, region_id=RegionOne) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:56.507244 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8349187-843e-4423-aaa7-70bae7c19be7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:56.508692 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8349187-843e-4423-aaa7-70bae7c19be7 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:56.513983 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 498/984] 10.222.0.13 () {56 vars in 1251 bytes} [Tue Jun 7 21:50:56 2022] GET /identity/v3/endpoints?service_id=5ad4118034e6440ead05b661f5708356&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:50:56.519244 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8349187-843e-4423-aaa7-70bae7c19be7 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:56.571206 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c27893-f043-473c-9224-077395fa5a05 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:56.605229 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfbc210-974e-4fd0-9182-bb08dbf46f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:56.605882 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfbc210-974e-4fd0-9182-bb08dbf46f29 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:56.606164 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfbc210-974e-4fd0-9182-bb08dbf46f29 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:56.606435 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfbc210-974e-4fd0-9182-bb08dbf46f29 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:56.607144 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfbc210-974e-4fd0-9182-bb08dbf46f29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:56.607480 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfbc210-974e-4fd0-9182-bb08dbf46f29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:56.608833 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfbc210-974e-4fd0-9182-bb08dbf46f29 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:56.624502 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 487/985] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:50:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:56.625461 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfbc210-974e-4fd0-9182-bb08dbf46f29 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:56.629309 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8349187-843e-4423-aaa7-70bae7c19be7 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:56.645610 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:56.664377 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c790194b-a5d1-4deb-b31d-5536d64075aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:56.665043 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790194b-a5d1-4deb-b31d-5536d64075aa None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:56.665334 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790194b-a5d1-4deb-b31d-5536d64075aa None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:56.665594 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790194b-a5d1-4deb-b31d-5536d64075aa None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:56.666281 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c790194b-a5d1-4deb-b31d-5536d64075aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:56.667673 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c790194b-a5d1-4deb-b31d-5536d64075aa None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:56.674035 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 499/986] 10.222.0.13 () {56 vars in 1087 bytes} [Tue Jun 7 21:50:56 2022] GET /identity/v3/services => generated 1727 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 21:50:56.750112 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfbc210-974e-4fd0-9182-bb08dbf46f29 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:56.796175 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97db6af-1c43-4d63-bbb2-78b16ddaf19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:56.796994 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97db6af-1c43-4d63-bbb2-78b16ddaf19d None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:56.797279 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97db6af-1c43-4d63-bbb2-78b16ddaf19d None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:56.797557 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97db6af-1c43-4d63-bbb2-78b16ddaf19d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:56.805326 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97db6af-1c43-4d63-bbb2-78b16ddaf19d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:56.806825 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97db6af-1c43-4d63-bbb2-78b16ddaf19d None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:56.823380 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-d97db6af-1c43-4d63-bbb2-78b16ddaf19d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:56.823380 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:56.823380 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:56.823380 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:56.823380 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:56.823380 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:56.823380 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:56.823380 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:56.823380 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:56.823380 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:56.823380 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:56.823380 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:56.823380 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 21:50:56.823380 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 21:50:56.823380 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 21:50:56.824359 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 21:50:56.824359 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:56.824359 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:56.824359 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:56.824359 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:56.824359 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:56.824359 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:56.824359 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:56.824359 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:56.824359 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:56.824359 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:56.824359 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:56.824359 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:56.824359 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:56.824359 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:56.824359 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:56.824359 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:56.825266 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 21:50:56.825266 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 21:50:56.825266 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 21:50:56.825266 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 21:50:56.825266 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 21:50:56.825266 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 21:50:56.825266 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:50:56.825266 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:56.827702 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 488/987] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:50:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:50:56.831153 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c790194b-a5d1-4deb-b31d-5536d64075aa None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:56.895409 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:56.940523 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c790194b-a5d1-4deb-b31d-5536d64075aa None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:56.974391 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb1d7e-acc4-4ac9-b404-158d6cc5d19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:56.975055 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb1d7e-acc4-4ac9-b404-158d6cc5d19e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:56.975357 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb1d7e-acc4-4ac9-b404-158d6cc5d19e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:56.975637 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb1d7e-acc4-4ac9-b404-158d6cc5d19e None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:56.976414 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cb1d7e-acc4-4ac9-b404-158d6cc5d19e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:56.976725 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cb1d7e-acc4-4ac9-b404-158d6cc5d19e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:56.978161 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cb1d7e-acc4-4ac9-b404-158d6cc5d19e None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:56.995814 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 500/988] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jun 7 21:50:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:56.997154 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97db6af-1c43-4d63-bbb2-78b16ddaf19d None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:57.076170 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:57.100357 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97db6af-1c43-4d63-bbb2-78b16ddaf19d None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:57.133893 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d2e13c-78d9-448a-a113-7a76954c4d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:57.134542 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2e13c-78d9-448a-a113-7a76954c4d28 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:57.134824 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2e13c-78d9-448a-a113-7a76954c4d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:57.135099 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2e13c-78d9-448a-a113-7a76954c4d28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:57.135957 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d2e13c-78d9-448a-a113-7a76954c4d28 None admin] RBAC: Adding query filter params (role.id=14b78e4dcfb84633981a9b2f2f06f4fb, scope.project.id=0219dbda54274fdc9bdccbd960c30c6b, user.id=86619a80df8841a3ba7d8471ac38056a) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:57.136267 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d2e13c-78d9-448a-a113-7a76954c4d28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:57.137682 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d2e13c-78d9-448a-a113-7a76954c4d28 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:57.144361 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 489/989] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:50:56 2022] GET /identity/v3/role_assignments?user.id=86619a80df8841a3ba7d8471ac38056a&scope.project.id=0219dbda54274fdc9bdccbd960c30c6b&role.id=14b78e4dcfb84633981a9b2f2f06f4fb => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:57.914538 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e693967-5eca-4519-a6e2-dd5137c55cc1 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:57.915049 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e693967-5eca-4519-a6e2-dd5137c55cc1 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:57.915159 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e693967-5eca-4519-a6e2-dd5137c55cc1 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:57.916520 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 501/990] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:57.921797 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287857f9-c4af-4587-88fe-3011429bd4e4 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:57.922197 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287857f9-c4af-4587-88fe-3011429bd4e4 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:57.922360 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287857f9-c4af-4587-88fe-3011429bd4e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:57.940189 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-287857f9-c4af-4587-88fe-3011429bd4e4 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:57.941390 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-287857f9-c4af-4587-88fe-3011429bd4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:58.009962 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 490/991] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:57 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:50:58.033184 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e693967-5eca-4519-a6e2-dd5137c55cc1 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:58.132792 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e693967-5eca-4519-a6e2-dd5137c55cc1 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:58.149512 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:58.167036 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6936c6e8-60fe-47b1-8949-bbe9859592be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:58.167872 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6936c6e8-60fe-47b1-8949-bbe9859592be None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:58.168156 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6936c6e8-60fe-47b1-8949-bbe9859592be None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:58.168431 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6936c6e8-60fe-47b1-8949-bbe9859592be None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:58.169079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6936c6e8-60fe-47b1-8949-bbe9859592be None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:58.170440 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6936c6e8-60fe-47b1-8949-bbe9859592be None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:58.183018 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-6936c6e8-60fe-47b1-8949-bbe9859592be None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 21:50:58.183018 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:58.183018 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 21:50:58.183018 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 21:50:58.183018 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 21:50:58.183018 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 21:50:58.183018 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 21:50:58.183018 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:50:58.183018 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 21:50:58.183018 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:58.183018 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 21:50:58.183018 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:58.183018 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:58.183018 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:58.183018 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:58.183018 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:58.183018 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:58.184203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:58.184203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:58.184203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:58.184203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:58.184203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:58.184203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:58.184203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 21:50:58.184203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 21:50:58.184203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 21:50:58.184203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 21:50:58.184203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:58.184203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:58.184203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:58.184203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:58.184203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:58.184203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:58.185046 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:58.185046 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:58.185046 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:58.185046 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:58.185046 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:58.185046 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:58.185046 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:58.185046 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:58.185046 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:58.185046 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:58.185046 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 21:50:58.185046 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:50:58.185046 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 21:50:58.185046 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:58.187649 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 502/992] 10.222.0.13 () {56 vars in 1126 bytes} [Tue Jun 7 21:50:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:58.190425 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287857f9-c4af-4587-88fe-3011429bd4e4 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:58.262692 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:58.300253 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287857f9-c4af-4587-88fe-3011429bd4e4 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:58.341774 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06715b82-bf4c-45ed-87c0-dfa72746a6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:58.342449 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06715b82-bf4c-45ed-87c0-dfa72746a6bc None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:58.342737 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06715b82-bf4c-45ed-87c0-dfa72746a6bc None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:58.343018 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06715b82-bf4c-45ed-87c0-dfa72746a6bc None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:58.343765 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06715b82-bf4c-45ed-87c0-dfa72746a6bc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:58.344078 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06715b82-bf4c-45ed-87c0-dfa72746a6bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:58.345976 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06715b82-bf4c-45ed-87c0-dfa72746a6bc None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:58.355391 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 491/993] 10.222.0.13 () {56 vars in 1122 bytes} [Tue Jun 7 21:50:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:58.357448 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6936c6e8-60fe-47b1-8949-bbe9859592be None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:58.460375 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6936c6e8-60fe-47b1-8949-bbe9859592be None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:58.497407 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d9ebd3-da86-4a27-8df8-84505bbd1e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:58.498073 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d9ebd3-da86-4a27-8df8-84505bbd1e29 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:58.498358 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d9ebd3-da86-4a27-8df8-84505bbd1e29 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:58.498637 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d9ebd3-da86-4a27-8df8-84505bbd1e29 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:58.499370 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d9ebd3-da86-4a27-8df8-84505bbd1e29 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:58.499682 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d9ebd3-da86-4a27-8df8-84505bbd1e29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:58.501157 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d9ebd3-da86-4a27-8df8-84505bbd1e29 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:58.506537 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 503/994] 10.222.0.13 () {56 vars in 1122 bytes} [Tue Jun 7 21:50:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:50:58.512142 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06715b82-bf4c-45ed-87c0-dfa72746a6bc None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:58.614634 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06715b82-bf4c-45ed-87c0-dfa72746a6bc None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:58.652220 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b938e1de-9cdb-4fe2-8c2c-62c309251817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:58.653007 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b938e1de-9cdb-4fe2-8c2c-62c309251817 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:58.653296 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b938e1de-9cdb-4fe2-8c2c-62c309251817 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:58.653630 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b938e1de-9cdb-4fe2-8c2c-62c309251817 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:58.654480 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b938e1de-9cdb-4fe2-8c2c-62c309251817 None admin] RBAC: Adding query filter params (interface=admin, service_id=b4ca70b9a07542b4a05ba279e5ff0fe5, region_id=RegionOne) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:58.654799 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b938e1de-9cdb-4fe2-8c2c-62c309251817 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:58.656284 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b938e1de-9cdb-4fe2-8c2c-62c309251817 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:58.658793 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:58.671446 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 492/995] 10.222.0.13 () {56 vars in 1249 bytes} [Tue Jun 7 21:50:58 2022] GET /identity/v3/endpoints?service_id=b4ca70b9a07542b4a05ba279e5ff0fe5&interface=admin®ion_id=RegionOne => generated 193 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:58.671788 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d9ebd3-da86-4a27-8df8-84505bbd1e29 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:58.722603 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:58.782330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d9ebd3-da86-4a27-8df8-84505bbd1e29 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:58.817071 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a561b29-cd48-40de-a0f0-76652e0e8821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:58.817845 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a561b29-cd48-40de-a0f0-76652e0e8821 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:58.818133 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a561b29-cd48-40de-a0f0-76652e0e8821 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:58.818411 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a561b29-cd48-40de-a0f0-76652e0e8821 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:58.819157 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a561b29-cd48-40de-a0f0-76652e0e8821 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:58.820566 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a561b29-cd48-40de-a0f0-76652e0e8821 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:58.834304 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 504/996] 10.222.0.13 () {56 vars in 1087 bytes} [Tue Jun 7 21:50:58 2022] GET /identity/v3/services => generated 1727 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 21:50:58.981876 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa827fb-4389-4e2e-95a8-c608c2c9fde9 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:58.982376 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa827fb-4389-4e2e-95a8-c608c2c9fde9 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:58.982376 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa827fb-4389-4e2e-95a8-c608c2c9fde9 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:58.983988 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 493/997] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:50:58.989913 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b69bfb-0201-4e99-9423-38858fca4a8a None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:58.990397 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b69bfb-0201-4e99-9423-38858fca4a8a None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:58.990472 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b69bfb-0201-4e99-9423-38858fca4a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:59.009172 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-14b69bfb-0201-4e99-9423-38858fca4a8a None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:59.010377 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-14b69bfb-0201-4e99-9423-38858fca4a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:59.084609 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 505/998] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:58 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:50:59.108008 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa827fb-4389-4e2e-95a8-c608c2c9fde9 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:59.214384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa827fb-4389-4e2e-95a8-c608c2c9fde9 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:59.248980 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eabdbd-e243-4edd-a2be-25423a892999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:59.249705 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eabdbd-e243-4edd-a2be-25423a892999 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:59.249987 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eabdbd-e243-4edd-a2be-25423a892999 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:59.250261 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eabdbd-e243-4edd-a2be-25423a892999 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:59.251005 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eabdbd-e243-4edd-a2be-25423a892999 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:59.252880 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eabdbd-e243-4edd-a2be-25423a892999 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:59.265210 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-77eabdbd-e243-4edd-a2be-25423a892999 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 21:50:59.265210 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:59.265210 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 21:50:59.265210 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 21:50:59.265210 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 21:50:59.265210 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 21:50:59.265210 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 21:50:59.265210 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:50:59.265210 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 21:50:59.265210 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:59.265210 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 21:50:59.265210 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:59.265210 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:59.265210 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:59.265210 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:59.265210 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:59.265210 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:59.266278 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:59.266278 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:59.266278 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:59.266278 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:59.266278 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:59.266278 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:59.266278 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 21:50:59.266278 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 21:50:59.266278 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 21:50:59.266278 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 21:50:59.266278 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:59.266278 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:59.266278 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:59.266278 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:59.266278 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:59.266278 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:59.269410 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:59.269410 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:59.269410 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:59.269410 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:59.269410 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:59.269410 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:59.269410 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:59.269410 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:59.269410 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:59.269410 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:59.269410 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 21:50:59.269410 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:50:59.269410 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 21:50:59.269410 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:50:59.271423 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 494/999] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:50:59 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:50:59.273359 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b69bfb-0201-4e99-9423-38858fca4a8a None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:59.317617 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:59.380315 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b69bfb-0201-4e99-9423-38858fca4a8a None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:59.414998 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f6cd5-9219-42cd-9fcd-b26ed86ce559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:59.415690 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f6cd5-9219-42cd-9fcd-b26ed86ce559 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:59.415973 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f6cd5-9219-42cd-9fcd-b26ed86ce559 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:59.416247 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f6cd5-9219-42cd-9fcd-b26ed86ce559 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:59.416950 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f6cd5-9219-42cd-9fcd-b26ed86ce559 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:59.417254 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f6cd5-9219-42cd-9fcd-b26ed86ce559 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:59.418612 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f6cd5-9219-42cd-9fcd-b26ed86ce559 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:59.423910 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 506/1000] 10.222.0.13 () {56 vars in 1112 bytes} [Tue Jun 7 21:50:59 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:50:59.429478 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eabdbd-e243-4edd-a2be-25423a892999 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:59.434477 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674d445-6a28-45fe-9dcb-12bb3bb5f0a1 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:59.434769 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674d445-6a28-45fe-9dcb-12bb3bb5f0a1 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:59.435046 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674d445-6a28-45fe-9dcb-12bb3bb5f0a1 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:59.436528 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 507/1001] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:50:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 21:50:59.441493 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb579a-45bf-4e9e-962e-f5fe3a78ec55 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:59.441777 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb579a-45bf-4e9e-962e-f5fe3a78ec55 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:59.442049 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb579a-45bf-4e9e-962e-f5fe3a78ec55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:59.461650 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-e4cb579a-45bf-4e9e-962e-f5fe3a78ec55 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:50:59.463069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-e4cb579a-45bf-4e9e-962e-f5fe3a78ec55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:50:59.498478 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:59.535066 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 508/1002] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:50:59 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 21:50:59.539389 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eabdbd-e243-4edd-a2be-25423a892999 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:59.557463 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cb579a-45bf-4e9e-962e-f5fe3a78ec55 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:59.578378 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f51afb5-2514-4179-b335-9e3547bea357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:59.579741 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f51afb5-2514-4179-b335-9e3547bea357 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:59.580066 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f51afb5-2514-4179-b335-9e3547bea357 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:59.580339 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f51afb5-2514-4179-b335-9e3547bea357 None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:59.581053 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f51afb5-2514-4179-b335-9e3547bea357 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:59.581358 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f51afb5-2514-4179-b335-9e3547bea357 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:59.582718 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f51afb5-2514-4179-b335-9e3547bea357 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:59.589084 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 495/1003] 10.222.0.13 () {56 vars in 1112 bytes} [Tue Jun 7 21:50:59 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:59.594844 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f51afb5-2514-4179-b335-9e3547bea357 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:59.598704 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:59.680313 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cb579a-45bf-4e9e-962e-f5fe3a78ec55 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:59.707454 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f51afb5-2514-4179-b335-9e3547bea357 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:59.719477 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30f8b98-5861-49bd-bf43-372a58a15e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:59.720628 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f8b98-5861-49bd-bf43-372a58a15e35 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:59.721100 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f8b98-5861-49bd-bf43-372a58a15e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:59.721789 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f8b98-5861-49bd-bf43-372a58a15e35 None admin] PATH_INFO: `/v3/services/image` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:59.723350 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30f8b98-5861-49bd-bf43-372a58a15e35 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:59.725422 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30f8b98-5861-49bd-bf43-372a58a15e35 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:59.734754 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:59.741733 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-a30f8b98-5861-49bd-bf43-372a58a15e35 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 21:50:59.741733 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:59.741733 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 21:50:59.741733 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 21:50:59.741733 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 21:50:59.741733 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 21:50:59.741733 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 21:50:59.741733 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:50:59.741733 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 21:50:59.741733 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:59.741733 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 21:50:59.741733 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:59.741733 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:50:59.741733 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:50:59.741733 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:50:59.741733 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:50:59.741733 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:50:59.742727 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:50:59.742727 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:50:59.742727 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:50:59.742727 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:50:59.742727 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:50:59.742727 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:50:59.742727 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 21:50:59.742727 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 21:50:59.742727 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 21:50:59.742727 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 21:50:59.742727 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:50:59.742727 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:50:59.742727 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:50:59.742727 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:50:59.742727 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:50:59.742727 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:50:59.743615 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:50:59.743615 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:50:59.743615 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:50:59.743615 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:50:59.743615 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:50:59.743615 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:50:59.743615 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:50:59.743615 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:50:59.743615 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:50:59.743615 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:50:59.743615 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 21:50:59.743615 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:50:59.743615 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 21:50:59.743615 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:50:59.744377 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c31ffc2-06ff-4c86-a9a9-0aab41a44bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:59.744377 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31ffc2-06ff-4c86-a9a9-0aab41a44bc8 None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:59.749815 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31ffc2-06ff-4c86-a9a9-0aab41a44bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:59.749938 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 509/1004] 10.222.0.13 () {56 vars in 1105 bytes} [Tue Jun 7 21:50:59 2022] GET /identity/v3/services/image => generated 86 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:50:59.750032 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31ffc2-06ff-4c86-a9a9-0aab41a44bc8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:59.750032 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c31ffc2-06ff-4c86-a9a9-0aab41a44bc8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:59.750032 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c31ffc2-06ff-4c86-a9a9-0aab41a44bc8 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:59.750431 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30f8b98-5861-49bd-bf43-372a58a15e35 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:50:59.777458 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 496/1005] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Jun 7 21:50:59 2022] POST /identity/v3/endpoints => generated 334 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 21:50:59.861280 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:50:59.867972 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30f8b98-5861-49bd-bf43-372a58a15e35 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:50:59.912577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4438dfe0-5777-4e8d-acbe-70de9be713f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:50:59.913258 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4438dfe0-5777-4e8d-acbe-70de9be713f8 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:50:59.913542 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4438dfe0-5777-4e8d-acbe-70de9be713f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:50:59.913811 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4438dfe0-5777-4e8d-acbe-70de9be713f8 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:50:59.914523 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4438dfe0-5777-4e8d-acbe-70de9be713f8 None admin] RBAC: Adding query filter params (name=image) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:50:59.914826 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4438dfe0-5777-4e8d-acbe-70de9be713f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:50:59.916251 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4438dfe0-5777-4e8d-acbe-70de9be713f8 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:50:59.928264 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 510/1006] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:59 2022] GET /identity/v3/services?name=image => generated 122 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:50:59.928470 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c31ffc2-06ff-4c86-a9a9-0aab41a44bc8 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:00.040757 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c31ffc2-06ff-4c86-a9a9-0aab41a44bc8 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:00.076036 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe48adab-4ef2-4193-aac1-3a80235648ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:00.076705 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe48adab-4ef2-4193-aac1-3a80235648ba None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:00.076991 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe48adab-4ef2-4193-aac1-3a80235648ba None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:00.077267 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe48adab-4ef2-4193-aac1-3a80235648ba None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:00.077972 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe48adab-4ef2-4193-aac1-3a80235648ba None admin] RBAC: Adding query filter params (type=image) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:51:00.078277 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe48adab-4ef2-4193-aac1-3a80235648ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:00.080188 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe48adab-4ef2-4193-aac1-3a80235648ba None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:00.084949 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 497/1007] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:50:59 2022] GET /identity/v3/services?type=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:01.154960 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd8ea8-0882-4915-ab71-f007dec05a6a None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:01.155517 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd8ea8-0882-4915-ab71-f007dec05a6a None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:01.155517 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd8ea8-0882-4915-ab71-f007dec05a6a None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:01.157418 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 511/1008] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:51:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:51:01.162887 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2055fd-25ed-4108-bea9-b8eeb981ef7c None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:01.163283 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2055fd-25ed-4108-bea9-b8eeb981ef7c None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:01.163472 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2055fd-25ed-4108-bea9-b8eeb981ef7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:01.180604 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-8c2055fd-25ed-4108-bea9-b8eeb981ef7c None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:51:01.181835 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-8c2055fd-25ed-4108-bea9-b8eeb981ef7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:51:01.226928 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:01.254000 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 498/1009] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:51:01 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:51:01.276822 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd8ea8-0882-4915-ab71-f007dec05a6a None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:01.386416 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd8ea8-0882-4915-ab71-f007dec05a6a None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:01.420553 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1468281-8fcf-4235-8194-5920e672a3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:01.421353 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1468281-8fcf-4235-8194-5920e672a3c2 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:01.421638 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1468281-8fcf-4235-8194-5920e672a3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:01.421913 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1468281-8fcf-4235-8194-5920e672a3c2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:01.422563 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1468281-8fcf-4235-8194-5920e672a3c2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:01.423962 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1468281-8fcf-4235-8194-5920e672a3c2 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:01.438631 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-c1468281-8fcf-4235-8194-5920e672a3c2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 21:51:01.438631 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:51:01.438631 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 21:51:01.438631 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 21:51:01.438631 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 21:51:01.438631 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 21:51:01.438631 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 21:51:01.438631 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:51:01.438631 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 21:51:01.438631 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:51:01.438631 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 21:51:01.438631 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:51:01.438631 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:51:01.438631 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:51:01.438631 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:51:01.438631 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:51:01.438631 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:51:01.439658 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:51:01.439658 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:51:01.439658 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:51:01.439658 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:51:01.439658 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:51:01.439658 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:51:01.439658 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 21:51:01.439658 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 21:51:01.439658 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 21:51:01.439658 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 21:51:01.439658 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:51:01.439658 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:51:01.439658 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:51:01.439658 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:51:01.439658 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:51:01.439658 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:51:01.440510 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:51:01.440510 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:51:01.440510 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:51:01.440510 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:51:01.440510 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:51:01.440510 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:51:01.440510 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:51:01.440510 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:51:01.440510 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:51:01.440510 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:51:01.440510 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 21:51:01.440510 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:51:01.440510 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 21:51:01.440510 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:51:01.440510 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 512/1010] 10.222.0.13 () {56 vars in 1126 bytes} [Tue Jun 7 21:51:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:51:01.446031 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2055fd-25ed-4108-bea9-b8eeb981ef7c None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:01.548023 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2055fd-25ed-4108-bea9-b8eeb981ef7c None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:01.582604 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d19d47-c58b-4436-9fd3-e887fb57f89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:01.583266 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d19d47-c58b-4436-9fd3-e887fb57f89f None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:01.583580 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d19d47-c58b-4436-9fd3-e887fb57f89f None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:01.583861 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d19d47-c58b-4436-9fd3-e887fb57f89f None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:01.584573 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d19d47-c58b-4436-9fd3-e887fb57f89f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:51:01.584881 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d19d47-c58b-4436-9fd3-e887fb57f89f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:01.586249 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d19d47-c58b-4436-9fd3-e887fb57f89f None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:01.596675 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 499/1011] 10.222.0.13 () {56 vars in 1122 bytes} [Tue Jun 7 21:51:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:01.597822 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1468281-8fcf-4235-8194-5920e672a3c2 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:01.623085 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:01.702376 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1468281-8fcf-4235-8194-5920e672a3c2 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:01.737254 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a8df8-4e64-4344-b334-ee647d42d1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:01.737922 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a8df8-4e64-4344-b334-ee647d42d1ac None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:01.738202 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a8df8-4e64-4344-b334-ee647d42d1ac None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:01.738476 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a8df8-4e64-4344-b334-ee647d42d1ac None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:01.739177 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837a8df8-4e64-4344-b334-ee647d42d1ac None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:51:01.739557 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837a8df8-4e64-4344-b334-ee647d42d1ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:01.740919 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837a8df8-4e64-4344-b334-ee647d42d1ac None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:01.746078 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 513/1012] 10.222.0.13 () {56 vars in 1122 bytes} [Tue Jun 7 21:51:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:51:01.752007 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d19d47-c58b-4436-9fd3-e887fb57f89f None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:01.802129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:01.867003 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d19d47-c58b-4436-9fd3-e887fb57f89f None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:01.903744 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484eeb47-4e9d-4747-bad5-a62f367a2cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:01.904402 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484eeb47-4e9d-4747-bad5-a62f367a2cc3 None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:01.904685 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484eeb47-4e9d-4747-bad5-a62f367a2cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:01.904960 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484eeb47-4e9d-4747-bad5-a62f367a2cc3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:01.905704 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484eeb47-4e9d-4747-bad5-a62f367a2cc3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:01.907119 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484eeb47-4e9d-4747-bad5-a62f367a2cc3 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:01.926199 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 500/1013] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Jun 7 21:51:01 2022] POST /identity/v3/endpoints => generated 325 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 21:51:02.193672 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f08d9-7a8f-4cfe-a57b-36294888daf6 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:02.194195 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f08d9-7a8f-4cfe-a57b-36294888daf6 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:02.194273 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f08d9-7a8f-4cfe-a57b-36294888daf6 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:02.195782 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 514/1014] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:51:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:51:02.200852 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b7700-3978-4834-949e-1ac5f234c077 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:02.201248 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b7700-3978-4834-949e-1ac5f234c077 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:02.201416 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b7700-3978-4834-949e-1ac5f234c077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:02.220079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-195b7700-3978-4834-949e-1ac5f234c077 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:51:02.221338 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-195b7700-3978-4834-949e-1ac5f234c077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:51:02.299371 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 501/1015] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:51:02 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:51:02.321974 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5f08d9-7a8f-4cfe-a57b-36294888daf6 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:02.381523 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf83d0-9e5f-4525-a5bf-20100b330fcc None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:02.381869 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf83d0-9e5f-4525-a5bf-20100b330fcc None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:02.382083 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf83d0-9e5f-4525-a5bf-20100b330fcc None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:02.383481 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 502/1016] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:51:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 21:51:02.388219 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d22173-2731-41fb-aa5e-06604ca9ee6d None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:02.388508 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d22173-2731-41fb-aa5e-06604ca9ee6d None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:02.388789 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d22173-2731-41fb-aa5e-06604ca9ee6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:02.405945 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-f8d22173-2731-41fb-aa5e-06604ca9ee6d None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:51:02.407358 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-f8d22173-2731-41fb-aa5e-06604ca9ee6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:51:02.419663 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:02.428810 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:02.442262 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5f08d9-7a8f-4cfe-a57b-36294888daf6 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:02.479480 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d54bf7c-77e7-4001-98f9-38a076076470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:02.480223 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d54bf7c-77e7-4001-98f9-38a076076470 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:02.480506 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d54bf7c-77e7-4001-98f9-38a076076470 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:02.480782 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d54bf7c-77e7-4001-98f9-38a076076470 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:02.482585 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 503/1017] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:51:02 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 21:51:02.492705 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d54bf7c-77e7-4001-98f9-38a076076470 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:51:02.501731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-3d54bf7c-77e7-4001-98f9-38a076076470 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 21:51:02.501731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:51:02.501731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:51:02.501731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:51:02.501731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:51:02.501731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:51:02.501731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:51:02.501731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:51:02.501731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:51:02.501731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:51:02.501731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:51:02.501731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:51:02.501731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:51:02.501731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:51:02.501731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:51:02.501731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:51:02.502684 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:51:02.502684 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:51:02.502684 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:51:02.502684 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:51:02.502684 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:51:02.502684 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:51:02.502684 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:51:02.502684 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:51:02.502684 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:51:02.502684 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:51:02.502684 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:51:02.502684 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:51:02.502684 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:51:02.502684 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:51:02.502684 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:51:02.502684 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:51:02.503570 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:51:02.503570 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:51:02.503570 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:51:02.503570 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:51:02.503570 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:51:02.503570 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:51:02.503570 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:51:02.503570 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:51:02.503570 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:51:02.503570 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:51:02.503570 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 21:51:02.503570 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:51:02.503570 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 515/1018] 10.222.0.13 () {56 vars in 1120 bytes} [Tue Jun 7 21:51:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:51:02.508347 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d22173-2731-41fb-aa5e-06604ca9ee6d None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:02.512056 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d54bf7c-77e7-4001-98f9-38a076076470 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:02.612244 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d22173-2731-41fb-aa5e-06604ca9ee6d None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:02.629872 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d54bf7c-77e7-4001-98f9-38a076076470 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:02.646940 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35476144-f45f-49d0-8b71-f2dfd856c55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:02.647626 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35476144-f45f-49d0-8b71-f2dfd856c55f None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:02.647910 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35476144-f45f-49d0-8b71-f2dfd856c55f None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:02.648188 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35476144-f45f-49d0-8b71-f2dfd856c55f None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:02.648939 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35476144-f45f-49d0-8b71-f2dfd856c55f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:02.650301 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35476144-f45f-49d0-8b71-f2dfd856c55f None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:02.665761 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 504/1019] 10.222.0.13 () {60 vars in 1141 bytes} [Tue Jun 7 21:51:02 2022] POST /identity/v3/services => generated 242 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 21:51:02.684581 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcf5ac6-c39c-480f-83ff-76d1bebb13de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:02.685357 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcf5ac6-c39c-480f-83ff-76d1bebb13de None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:02.685651 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcf5ac6-c39c-480f-83ff-76d1bebb13de None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:02.685931 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcf5ac6-c39c-480f-83ff-76d1bebb13de None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:02.686718 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcf5ac6-c39c-480f-83ff-76d1bebb13de None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:51:02.687037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcf5ac6-c39c-480f-83ff-76d1bebb13de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:02.688628 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcf5ac6-c39c-480f-83ff-76d1bebb13de None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:02.698732 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 516/1020] 10.222.0.13 () {56 vars in 1115 bytes} [Tue Jun 7 21:51:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:51:02.705907 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35476144-f45f-49d0-8b71-f2dfd856c55f None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:02.728762 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:02.832850 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35476144-f45f-49d0-8b71-f2dfd856c55f None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:02.870944 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a762c01a-3584-4ff4-a883-535da1f4e1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:02.872020 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a762c01a-3584-4ff4-a883-535da1f4e1a1 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:02.872472 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a762c01a-3584-4ff4-a883-535da1f4e1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:02.872899 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a762c01a-3584-4ff4-a883-535da1f4e1a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:02.884588 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a762c01a-3584-4ff4-a883-535da1f4e1a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:02.886206 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a762c01a-3584-4ff4-a883-535da1f4e1a1 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:02.904123 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-a762c01a-3584-4ff4-a883-535da1f4e1a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:51:02.904123 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:51:02.904123 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:51:02.904123 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:51:02.904123 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:51:02.904123 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:51:02.904123 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:51:02.904123 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:51:02.904123 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:51:02.904123 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:51:02.904123 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:51:02.904123 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:51:02.904123 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:51:02.904123 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:51:02.904123 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:51:02.905046 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:51:02.905046 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:51:02.905046 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:51:02.905046 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:51:02.905046 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:51:02.905046 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:51:02.905046 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:51:02.905046 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:51:02.905046 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:51:02.905046 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:51:02.905046 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:51:02.905046 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:51:02.905046 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:51:02.905046 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:51:02.905046 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:51:02.905046 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:51:02.905046 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:51:02.906026 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:51:02.906026 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:51:02.906026 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:51:02.906026 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:51:02.906026 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 505/1021] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:51:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:51:02.912843 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcf5ac6-c39c-480f-83ff-76d1bebb13de None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:02.916724 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:03.015512 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcf5ac6-c39c-480f-83ff-76d1bebb13de None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:03.051654 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2be8e9-bd34-447d-b20e-2339c9f948be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:03.052318 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2be8e9-bd34-447d-b20e-2339c9f948be None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:03.052597 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2be8e9-bd34-447d-b20e-2339c9f948be None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:03.052869 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2be8e9-bd34-447d-b20e-2339c9f948be None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:03.053702 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2be8e9-bd34-447d-b20e-2339c9f948be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:51:03.054005 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2be8e9-bd34-447d-b20e-2339c9f948be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:03.055388 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2be8e9-bd34-447d-b20e-2339c9f948be None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:03.073371 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 517/1022] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:51:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:51:03.074885 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a762c01a-3584-4ff4-a883-535da1f4e1a1 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:03.146321 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:03.182478 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a762c01a-3584-4ff4-a883-535da1f4e1a1 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:03.216840 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc0c69-8562-4a9b-985d-8e7d2975b2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:03.219453 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc0c69-8562-4a9b-985d-8e7d2975b2e3 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:03.219453 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc0c69-8562-4a9b-985d-8e7d2975b2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:03.219453 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc0c69-8562-4a9b-985d-8e7d2975b2e3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:03.227147 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dc0c69-8562-4a9b-985d-8e7d2975b2e3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:03.228630 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dc0c69-8562-4a9b-985d-8e7d2975b2e3 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:03.242730 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-81dc0c69-8562-4a9b-985d-8e7d2975b2e3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 21:51:03.242730 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:51:03.242730 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:51:03.242730 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:51:03.242730 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:51:03.242730 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:51:03.242730 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:51:03.242730 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:51:03.242730 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:51:03.242730 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:51:03.242730 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:51:03.242730 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:51:03.242730 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 21:51:03.242730 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 21:51:03.242730 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 21:51:03.242730 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 21:51:03.243900 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:51:03.243900 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:51:03.243900 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 21:51:03.243900 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:51:03.243900 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 21:51:03.243900 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:51:03.243900 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:51:03.243900 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:51:03.243900 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:51:03.243900 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:51:03.243900 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:51:03.243900 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:51:03.243900 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:51:03.243900 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:51:03.243900 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:51:03.243900 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:51:03.243900 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:51:03.243900 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:51:03.244830 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:51:03.244830 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:51:03.244830 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 21:51:03.244830 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 21:51:03.244830 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 21:51:03.244830 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 21:51:03.244830 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 21:51:03.244830 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 21:51:03.244830 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 21:51:03.244830 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:51:03.247641 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 506/1023] 10.222.0.13 () {56 vars in 1093 bytes} [Tue Jun 7 21:51:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:51:03.249642 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2be8e9-bd34-447d-b20e-2339c9f948be None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:03.312979 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:03.352519 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2be8e9-bd34-447d-b20e-2339c9f948be None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:03.387367 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60052bb-f198-4990-8e68-5e90eaa75be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:03.388065 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60052bb-f198-4990-8e68-5e90eaa75be2 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:03.388352 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60052bb-f198-4990-8e68-5e90eaa75be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:03.388630 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60052bb-f198-4990-8e68-5e90eaa75be2 None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:03.389468 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60052bb-f198-4990-8e68-5e90eaa75be2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:51:03.389780 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60052bb-f198-4990-8e68-5e90eaa75be2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:03.391376 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60052bb-f198-4990-8e68-5e90eaa75be2 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:03.414102 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 518/1024] 10.222.0.13 () {56 vars in 1133 bytes} [Tue Jun 7 21:51:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:51:03.415365 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc0c69-8562-4a9b-985d-8e7d2975b2e3 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:03.527898 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc0c69-8562-4a9b-985d-8e7d2975b2e3 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:03.555950 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:03.565714 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d290ee9-638e-4922-8230-0c8afbb01084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:03.566293 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d290ee9-638e-4922-8230-0c8afbb01084 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:03.566575 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d290ee9-638e-4922-8230-0c8afbb01084 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:03.566849 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d290ee9-638e-4922-8230-0c8afbb01084 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:03.637181 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d290ee9-638e-4922-8230-0c8afbb01084 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:03.638622 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d290ee9-638e-4922-8230-0c8afbb01084 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:03.743146 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 507/1025] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:51:03 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 21:51:03.744412 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60052bb-f198-4990-8e68-5e90eaa75be2 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:03.854750 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60052bb-f198-4990-8e68-5e90eaa75be2 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:03.878163 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:03.891983 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f5b8fd-848b-4b8c-a64b-dd42ddebfd33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:03.892785 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5b8fd-848b-4b8c-a64b-dd42ddebfd33 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:03.893059 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5b8fd-848b-4b8c-a64b-dd42ddebfd33 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:03.893327 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5b8fd-848b-4b8c-a64b-dd42ddebfd33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:03.903658 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f5b8fd-848b-4b8c-a64b-dd42ddebfd33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:03.905098 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f5b8fd-848b-4b8c-a64b-dd42ddebfd33 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:03.923301 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-75f5b8fd-848b-4b8c-a64b-dd42ddebfd33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:51:03.923301 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:51:03.923301 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:51:03.923301 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:51:03.923301 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:51:03.923301 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:51:03.923301 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:51:03.923301 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:51:03.923301 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:51:03.923301 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:51:03.923301 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:51:03.923301 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:51:03.923301 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:51:03.923301 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:51:03.923301 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:51:03.930179 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:51:03.930179 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:51:03.930179 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:51:03.930179 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:51:03.930179 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:51:03.930179 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:51:03.930179 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:51:03.930179 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:51:03.930179 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:51:03.930179 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:51:03.930179 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:51:03.930179 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:51:03.930179 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:51:03.930179 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:51:03.930179 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:51:03.930179 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:51:03.930179 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:51:03.931097 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:51:03.931097 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:51:03.931097 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:51:03.931097 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:51:03.931097 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 519/1026] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:51:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:51:03.932865 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d290ee9-638e-4922-8230-0c8afbb01084 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:03.964871 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:04.042956 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d290ee9-638e-4922-8230-0c8afbb01084 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:04.076798 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e7d13a-f61b-45e9-a23c-ee0864a3e832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:04.077455 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7d13a-f61b-45e9-a23c-ee0864a3e832 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:04.077737 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7d13a-f61b-45e9-a23c-ee0864a3e832 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:04.078014 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7d13a-f61b-45e9-a23c-ee0864a3e832 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:04.078779 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e7d13a-f61b-45e9-a23c-ee0864a3e832 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:51:04.079582 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e7d13a-f61b-45e9-a23c-ee0864a3e832 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:04.080941 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e7d13a-f61b-45e9-a23c-ee0864a3e832 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:04.099003 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 508/1027] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:51:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:04.100283 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f5b8fd-848b-4b8c-a64b-dd42ddebfd33 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:04.200643 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f5b8fd-848b-4b8c-a64b-dd42ddebfd33 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:04.234065 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45da4b70-4c04-4ca0-9d3a-430b0a58a15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:04.234861 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da4b70-4c04-4ca0-9d3a-430b0a58a15c None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:04.235356 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da4b70-4c04-4ca0-9d3a-430b0a58a15c None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:04.235693 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da4b70-4c04-4ca0-9d3a-430b0a58a15c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:04.242447 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45da4b70-4c04-4ca0-9d3a-430b0a58a15c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:04.243925 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45da4b70-4c04-4ca0-9d3a-430b0a58a15c None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:04.258462 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-45da4b70-4c04-4ca0-9d3a-430b0a58a15c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:51:04.258462 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:51:04.258462 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:51:04.258462 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:51:04.258462 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:51:04.258462 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:51:04.258462 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:51:04.258462 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:51:04.258462 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:51:04.258462 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:51:04.258462 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:51:04.258462 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:51:04.258462 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 21:51:04.258462 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 21:51:04.258462 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 21:51:04.264746 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 21:51:04.264746 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:51:04.264746 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:51:04.264746 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:51:04.264746 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:51:04.264746 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:51:04.264746 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:51:04.264746 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:51:04.264746 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:51:04.264746 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:51:04.264746 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:51:04.264746 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:51:04.264746 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:51:04.264746 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:51:04.264746 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:51:04.264746 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:51:04.264746 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:51:04.265864 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 21:51:04.265864 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 21:51:04.265864 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 21:51:04.265864 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 21:51:04.265864 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 21:51:04.265864 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 21:51:04.265864 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:51:04.265864 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:51:04.265864 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 520/1028] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:51:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:51:04.268450 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e7d13a-f61b-45e9-a23c-ee0864a3e832 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:04.295482 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a83bd4-565c-48b7-8030-d8232299d3df None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:04.295775 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a83bd4-565c-48b7-8030-d8232299d3df None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:04.296050 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a83bd4-565c-48b7-8030-d8232299d3df None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:04.297433 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 521/1029] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:51:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 21:51:04.302406 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea17b3b-126e-421c-9714-04d6f54d7a17 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:04.302689 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea17b3b-126e-421c-9714-04d6f54d7a17 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:04.302962 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea17b3b-126e-421c-9714-04d6f54d7a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:04.321459 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-aea17b3b-126e-421c-9714-04d6f54d7a17 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:51:04.322740 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-aea17b3b-126e-421c-9714-04d6f54d7a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:51:04.351314 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:04.365780 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:04.379977 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e7d13a-f61b-45e9-a23c-ee0864a3e832 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:04.396306 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 522/1030] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:51:04 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 21:51:04.419971 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f1f7ea-431f-43c4-9be0-4ee6c1c5f5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:04.420637 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f1f7ea-431f-43c4-9be0-4ee6c1c5f5fa None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:04.420916 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f1f7ea-431f-43c4-9be0-4ee6c1c5f5fa None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:04.421189 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f1f7ea-431f-43c4-9be0-4ee6c1c5f5fa None admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:04.422195 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea17b3b-126e-421c-9714-04d6f54d7a17 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:04.424288 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f1f7ea-431f-43c4-9be0-4ee6c1c5f5fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:51:04.424741 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f1f7ea-431f-43c4-9be0-4ee6c1c5f5fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:04.426190 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f1f7ea-431f-43c4-9be0-4ee6c1c5f5fa None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:04.444946 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 509/1031] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jun 7 21:51:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:04.445962 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f1f7ea-431f-43c4-9be0-4ee6c1c5f5fa None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:04.535955 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea17b3b-126e-421c-9714-04d6f54d7a17 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:04.557930 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f1f7ea-431f-43c4-9be0-4ee6c1c5f5fa None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:04.574818 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8726c1-37be-4dc6-9b07-c9e23a037696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:04.575631 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8726c1-37be-4dc6-9b07-c9e23a037696 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:04.575954 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8726c1-37be-4dc6-9b07-c9e23a037696 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:04.576170 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8726c1-37be-4dc6-9b07-c9e23a037696 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:04.577335 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8726c1-37be-4dc6-9b07-c9e23a037696 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:04.578793 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8726c1-37be-4dc6-9b07-c9e23a037696 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:04.581035 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 523/1032] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:51:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:51:04.586713 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8726c1-37be-4dc6-9b07-c9e23a037696 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:04.594997 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c11bd16-ab0f-493d-9f53-a0897af1ca6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:04.596188 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11bd16-ab0f-493d-9f53-a0897af1ca6d None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:04.596479 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11bd16-ab0f-493d-9f53-a0897af1ca6d None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:04.596757 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11bd16-ab0f-493d-9f53-a0897af1ca6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:04.598200 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c11bd16-ab0f-493d-9f53-a0897af1ca6d None admin] RBAC: Adding query filter params (role.id=14b78e4dcfb84633981a9b2f2f06f4fb, scope.project.id=0219dbda54274fdc9bdccbd960c30c6b, user.id=dbe0303ea0764347bfb054870ebba41f) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:51:04.598524 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c11bd16-ab0f-493d-9f53-a0897af1ca6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:04.600324 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c11bd16-ab0f-493d-9f53-a0897af1ca6d None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:04.607085 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 510/1033] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:51:04 2022] GET /identity/v3/role_assignments?user.id=dbe0303ea0764347bfb054870ebba41f&scope.project.id=0219dbda54274fdc9bdccbd960c30c6b&role.id=14b78e4dcfb84633981a9b2f2f06f4fb => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:51:04.623651 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:04.712223 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8726c1-37be-4dc6-9b07-c9e23a037696 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:04.757045 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431f6db2-29e2-4761-bc51-217ecf7b9831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:04.757749 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f6db2-29e2-4761-bc51-217ecf7b9831 None admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:04.758013 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f6db2-29e2-4761-bc51-217ecf7b9831 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:04.758289 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f6db2-29e2-4761-bc51-217ecf7b9831 None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:04.759010 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431f6db2-29e2-4761-bc51-217ecf7b9831 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:04.760526 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431f6db2-29e2-4761-bc51-217ecf7b9831 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:04.794515 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 524/1034] 10.222.0.13 () {60 vars in 1141 bytes} [Tue Jun 7 21:51:04 2022] POST /identity/v3/projects => generated 311 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 21:51:05.031357 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb856c7c-e975-400a-afa5-97235bbb8a66 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:05.031713 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb856c7c-e975-400a-afa5-97235bbb8a66 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:05.032616 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb856c7c-e975-400a-afa5-97235bbb8a66 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:05.034923 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 511/1035] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:51:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:51:05.040352 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aabb87f-2433-4667-ae01-94759e46fa67 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:05.040650 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aabb87f-2433-4667-ae01-94759e46fa67 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:05.040936 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aabb87f-2433-4667-ae01-94759e46fa67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:05.059561 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-8aabb87f-2433-4667-ae01-94759e46fa67 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:51:05.060682 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-8aabb87f-2433-4667-ae01-94759e46fa67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:51:05.132080 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 525/1036] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:51:05 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:51:05.155566 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb856c7c-e975-400a-afa5-97235bbb8a66 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:05.170686 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:05.261375 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb856c7c-e975-400a-afa5-97235bbb8a66 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:05.295205 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1633c43e-52ab-4510-b259-173a8cf04cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:05.296013 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1633c43e-52ab-4510-b259-173a8cf04cc7 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:05.296293 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1633c43e-52ab-4510-b259-173a8cf04cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:05.296568 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1633c43e-52ab-4510-b259-173a8cf04cc7 None admin] PATH_INFO: `/v3/services/image` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:05.297215 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1633c43e-52ab-4510-b259-173a8cf04cc7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:05.299095 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1633c43e-52ab-4510-b259-173a8cf04cc7 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:05.310615 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-1633c43e-52ab-4510-b259-173a8cf04cc7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 21:51:05.310615 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:51:05.310615 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 21:51:05.310615 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 21:51:05.310615 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 21:51:05.310615 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 21:51:05.310615 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 21:51:05.310615 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:51:05.310615 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 21:51:05.310615 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:51:05.310615 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 21:51:05.310615 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:51:05.310615 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:51:05.310615 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:51:05.310615 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:51:05.310615 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:51:05.310615 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:51:05.311869 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:51:05.311869 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:51:05.311869 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:51:05.311869 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:51:05.311869 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:51:05.311869 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:51:05.311869 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 21:51:05.311869 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 21:51:05.311869 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 21:51:05.311869 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 21:51:05.311869 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:51:05.311869 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:51:05.311869 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:51:05.311869 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:51:05.311869 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:51:05.311869 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:51:05.312719 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:51:05.312719 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:51:05.312719 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:51:05.312719 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:51:05.312719 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:51:05.312719 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:51:05.312719 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:51:05.312719 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:51:05.312719 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:51:05.312719 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:51:05.312719 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 21:51:05.312719 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:51:05.312719 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 21:51:05.312719 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:51:05.314039 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 512/1037] 10.222.0.13 () {56 vars in 1105 bytes} [Tue Jun 7 21:51:05 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:51:05.317787 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aabb87f-2433-4667-ae01-94759e46fa67 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:05.334180 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:05.422500 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aabb87f-2433-4667-ae01-94759e46fa67 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:05.456842 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142094df-1446-4816-896b-43a15896c750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:05.457513 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142094df-1446-4816-896b-43a15896c750 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:05.457794 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142094df-1446-4816-896b-43a15896c750 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:05.458071 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142094df-1446-4816-896b-43a15896c750 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:05.458778 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142094df-1446-4816-896b-43a15896c750 None admin] RBAC: Adding query filter params (name=image) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:51:05.459083 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142094df-1446-4816-896b-43a15896c750 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:05.460508 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142094df-1446-4816-896b-43a15896c750 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:05.465995 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 526/1038] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:51:05 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:05.472548 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1633c43e-52ab-4510-b259-173a8cf04cc7 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:05.550287 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:05.574418 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1633c43e-52ab-4510-b259-173a8cf04cc7 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:05.607781 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e41488b-3065-494a-b95c-e118f7342b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:05.608447 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41488b-3065-494a-b95c-e118f7342b71 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:05.608728 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41488b-3065-494a-b95c-e118f7342b71 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:05.609007 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41488b-3065-494a-b95c-e118f7342b71 None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:05.609711 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e41488b-3065-494a-b95c-e118f7342b71 None admin] RBAC: Adding query filter params (type=image) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:51:05.610054 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e41488b-3065-494a-b95c-e118f7342b71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:05.611435 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e41488b-3065-494a-b95c-e118f7342b71 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:05.616589 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 513/1039] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:51:05 2022] GET /identity/v3/services?type=image => generated 350 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:05.622214 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142094df-1446-4816-896b-43a15896c750 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:05.702520 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:05.725668 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142094df-1446-4816-896b-43a15896c750 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:05.764367 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0528e80-ba08-451d-ae84-95e5557231a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:05.765276 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0528e80-ba08-451d-ae84-95e5557231a2 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:05.765756 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0528e80-ba08-451d-ae84-95e5557231a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:05.766995 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0528e80-ba08-451d-ae84-95e5557231a2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:05.768104 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0528e80-ba08-451d-ae84-95e5557231a2 None admin] RBAC: Adding query filter params (interface=public, service_id=e9a235a9c0cf4689a2548aefc4e55b13, region_id=RegionOne) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:51:05.768617 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0528e80-ba08-451d-ae84-95e5557231a2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:05.770220 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0528e80-ba08-451d-ae84-95e5557231a2 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:05.779901 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 527/1040] 10.222.0.13 () {56 vars in 1251 bytes} [Tue Jun 7 21:51:05 2022] GET /identity/v3/endpoints?service_id=e9a235a9c0cf4689a2548aefc4e55b13&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:51:05.781129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e41488b-3065-494a-b95c-e118f7342b71 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:05.887769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e41488b-3065-494a-b95c-e118f7342b71 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:05.921227 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428261c5-830a-4ee5-a70e-3a7b242c766a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:05.921894 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428261c5-830a-4ee5-a70e-3a7b242c766a None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:05.922179 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428261c5-830a-4ee5-a70e-3a7b242c766a None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:05.923024 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428261c5-830a-4ee5-a70e-3a7b242c766a None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:05.923901 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428261c5-830a-4ee5-a70e-3a7b242c766a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:05.925285 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428261c5-830a-4ee5-a70e-3a7b242c766a None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:05.939363 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 514/1041] 10.222.0.13 () {56 vars in 1087 bytes} [Tue Jun 7 21:51:05 2022] GET /identity/v3/services => generated 1957 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 21:51:06.882591 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a203c-6257-4b20-8f3d-12ccfcd4d3de None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:06.883125 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a203c-6257-4b20-8f3d-12ccfcd4d3de None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:06.883125 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a203c-6257-4b20-8f3d-12ccfcd4d3de None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:06.884673 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 528/1042] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:51:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:51:06.889669 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5731b8-9fc0-4cd2-a36c-c14b8a0a0761 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:06.890101 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5731b8-9fc0-4cd2-a36c-c14b8a0a0761 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:06.890258 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5731b8-9fc0-4cd2-a36c-c14b8a0a0761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:06.897091 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:06.908121 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-fc5731b8-9fc0-4cd2-a36c-c14b8a0a0761 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:51:06.909482 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-fc5731b8-9fc0-4cd2-a36c-c14b8a0a0761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:51:06.982452 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 515/1043] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:51:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:51:07.006550 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088a203c-6257-4b20-8f3d-12ccfcd4d3de None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:07.115525 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088a203c-6257-4b20-8f3d-12ccfcd4d3de None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:07.138237 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0fc14-3e1b-4b4c-81a7-423b10a925be None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:07.138514 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0fc14-3e1b-4b4c-81a7-423b10a925be None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:07.138794 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0fc14-3e1b-4b4c-81a7-423b10a925be None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:07.140200 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 516/1044] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:51:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 21:51:07.145305 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d12b1-0957-43dd-b14f-8abe6e39da2c None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:07.145593 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d12b1-0957-43dd-b14f-8abe6e39da2c None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:07.145862 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d12b1-0957-43dd-b14f-8abe6e39da2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:07.152433 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6172fc-fde1-4e2b-a8c6-532ac3ffaf78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:07.153244 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6172fc-fde1-4e2b-a8c6-532ac3ffaf78 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:07.153529 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6172fc-fde1-4e2b-a8c6-532ac3ffaf78 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:07.153806 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6172fc-fde1-4e2b-a8c6-532ac3ffaf78 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:07.163315 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-1f9d12b1-0957-43dd-b14f-8abe6e39da2c None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:51:07.164523 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-1f9d12b1-0957-43dd-b14f-8abe6e39da2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:51:07.164784 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6172fc-fde1-4e2b-a8c6-532ac3ffaf78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:51:07.172977 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-8e6172fc-fde1-4e2b-a8c6-532ac3ffaf78 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 21:51:07.172977 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:51:07.172977 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:51:07.172977 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:51:07.172977 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:51:07.172977 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:51:07.172977 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:51:07.172977 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:51:07.172977 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:51:07.172977 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:51:07.172977 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:51:07.172977 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:51:07.172977 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:51:07.172977 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:51:07.172977 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:51:07.172977 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:51:07.173918 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:51:07.173918 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:51:07.173918 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:51:07.173918 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:51:07.173918 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:51:07.173918 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:51:07.173918 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:51:07.173918 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:51:07.173918 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:51:07.173918 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:51:07.173918 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:51:07.173918 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:51:07.173918 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:51:07.173918 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:51:07.173918 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:51:07.173918 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:51:07.174781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:51:07.174781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:51:07.174781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:51:07.174781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:51:07.174781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:51:07.174781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:51:07.174781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:51:07.174781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:51:07.174781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:51:07.174781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:51:07.174781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 21:51:07.174781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:51:07.174781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 529/1045] 10.222.0.13 () {56 vars in 1120 bytes} [Tue Jun 7 21:51:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:51:07.179976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6172fc-fde1-4e2b-a8c6-532ac3ffaf78 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:07.182784 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:07.240881 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 517/1046] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:51:07 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 21:51:07.265087 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9d12b1-0957-43dd-b14f-8abe6e39da2c None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:07.291556 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6172fc-fde1-4e2b-a8c6-532ac3ffaf78 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:07.326622 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87515d4e-1eb4-4efc-9c44-61c3a79d3ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:07.327293 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87515d4e-1eb4-4efc-9c44-61c3a79d3ee3 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:07.327597 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87515d4e-1eb4-4efc-9c44-61c3a79d3ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:07.327871 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87515d4e-1eb4-4efc-9c44-61c3a79d3ee3 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:07.328754 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87515d4e-1eb4-4efc-9c44-61c3a79d3ee3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:51:07.329296 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87515d4e-1eb4-4efc-9c44-61c3a79d3ee3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:07.330895 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:07.331026 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87515d4e-1eb4-4efc-9c44-61c3a79d3ee3 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:07.339353 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 530/1047] 10.222.0.13 () {56 vars in 1115 bytes} [Tue Jun 7 21:51:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:51:07.345232 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87515d4e-1eb4-4efc-9c44-61c3a79d3ee3 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:07.374320 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9d12b1-0957-43dd-b14f-8abe6e39da2c None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:07.413278 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d025d031-eaef-4814-983a-48c77f9f14d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:07.414065 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025d031-eaef-4814-983a-48c77f9f14d2 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:07.414350 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025d031-eaef-4814-983a-48c77f9f14d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:07.414627 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025d031-eaef-4814-983a-48c77f9f14d2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:07.415894 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d025d031-eaef-4814-983a-48c77f9f14d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:07.417337 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d025d031-eaef-4814-983a-48c77f9f14d2 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:07.419258 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 518/1048] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:51:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:51:07.425238 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d025d031-eaef-4814-983a-48c77f9f14d2 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:07.426459 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:07.480333 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87515d4e-1eb4-4efc-9c44-61c3a79d3ee3 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:07.513811 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621ba786-fd51-42c3-981f-bb33b5c15f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:07.514590 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ba786-fd51-42c3-981f-bb33b5c15f89 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:07.514877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ba786-fd51-42c3-981f-bb33b5c15f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:07.515150 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ba786-fd51-42c3-981f-bb33b5c15f89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:07.526266 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621ba786-fd51-42c3-981f-bb33b5c15f89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:07.527742 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621ba786-fd51-42c3-981f-bb33b5c15f89 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:07.544793 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-621ba786-fd51-42c3-981f-bb33b5c15f89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:51:07.544793 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:51:07.544793 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:51:07.544793 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:51:07.544793 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:51:07.544793 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:51:07.544793 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:51:07.544793 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:51:07.544793 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:51:07.544793 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:51:07.544793 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:51:07.544793 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:51:07.544793 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:51:07.544793 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:51:07.544793 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:51:07.545732 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:51:07.545732 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:51:07.545732 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:51:07.545732 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:51:07.545732 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:51:07.545732 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:51:07.545732 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:51:07.545732 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:51:07.545732 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:51:07.545732 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:51:07.545732 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:51:07.545732 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:51:07.545732 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:51:07.545732 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:51:07.545732 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:51:07.545732 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:51:07.545732 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:51:07.547885 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:51:07.547885 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:51:07.547885 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:51:07.547885 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:51:07.547885 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 531/1049] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:51:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:51:07.552134 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621ba786-fd51-42c3-981f-bb33b5c15f89 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:07.553243 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d025d031-eaef-4814-983a-48c77f9f14d2 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:07.585556 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:07.591534 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d209c4e-6295-4648-aa03-72a103c5ab98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:07.592215 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d209c4e-6295-4648-aa03-72a103c5ab98 None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:07.592497 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d209c4e-6295-4648-aa03-72a103c5ab98 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:07.592771 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d209c4e-6295-4648-aa03-72a103c5ab98 None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:07.593484 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d209c4e-6295-4648-aa03-72a103c5ab98 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:07.595161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d209c4e-6295-4648-aa03-72a103c5ab98 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:07.627471 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 519/1050] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Jun 7 21:51:07 2022] POST /identity/v3/users => generated 285 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 21:51:07.678019 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621ba786-fd51-42c3-981f-bb33b5c15f89 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:07.696843 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:07.717566 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668300e5-bc46-4688-988f-4849104beaf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:07.718250 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668300e5-bc46-4688-988f-4849104beaf2 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:07.718536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668300e5-bc46-4688-988f-4849104beaf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:07.718812 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668300e5-bc46-4688-988f-4849104beaf2 None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:07.719637 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668300e5-bc46-4688-988f-4849104beaf2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:51:07.720071 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668300e5-bc46-4688-988f-4849104beaf2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:07.721476 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668300e5-bc46-4688-988f-4849104beaf2 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:07.736341 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 532/1051] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:51:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:51:07.741700 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d209c4e-6295-4648-aa03-72a103c5ab98 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:07.856302 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d209c4e-6295-4648-aa03-72a103c5ab98 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:07.895470 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a73cfe-afaa-4a60-97fc-cb4cc1e74dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:07.896204 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a73cfe-afaa-4a60-97fc-cb4cc1e74dae None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:07.896489 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a73cfe-afaa-4a60-97fc-cb4cc1e74dae None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:07.896765 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a73cfe-afaa-4a60-97fc-cb4cc1e74dae None admin] PATH_INFO: `/v3/users/nova` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:07.905535 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a73cfe-afaa-4a60-97fc-cb4cc1e74dae None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:07.907520 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a73cfe-afaa-4a60-97fc-cb4cc1e74dae None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:07.924158 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-09a73cfe-afaa-4a60-97fc-cb4cc1e74dae None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 21:51:07.924158 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:51:07.924158 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:51:07.924158 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:51:07.924158 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:51:07.924158 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:51:07.924158 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:51:07.924158 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:51:07.924158 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:51:07.924158 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:51:07.924158 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:51:07.924158 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:51:07.924158 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 21:51:07.924158 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 21:51:07.924158 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 21:51:07.924158 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 21:51:07.925175 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:51:07.925175 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:51:07.925175 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 21:51:07.925175 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:51:07.925175 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 21:51:07.925175 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:51:07.925175 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:51:07.925175 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:51:07.925175 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:51:07.925175 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:51:07.925175 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:51:07.925175 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:51:07.925175 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:51:07.925175 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:51:07.925175 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:51:07.925175 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:51:07.925175 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:51:07.925175 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:51:07.926132 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:51:07.926132 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:51:07.926132 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 21:51:07.926132 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 21:51:07.926132 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 21:51:07.926132 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 21:51:07.926132 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 21:51:07.926132 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 21:51:07.926132 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 21:51:07.926132 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:51:07.926132 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 520/1052] 10.222.0.13 () {56 vars in 1093 bytes} [Tue Jun 7 21:51:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:51:07.931363 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668300e5-bc46-4688-988f-4849104beaf2 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:08.035129 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668300e5-bc46-4688-988f-4849104beaf2 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:08.069070 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6773997-95e9-4886-8312-336685769500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:08.069752 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6773997-95e9-4886-8312-336685769500 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:08.070033 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6773997-95e9-4886-8312-336685769500 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:08.070309 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6773997-95e9-4886-8312-336685769500 None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:08.071141 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6773997-95e9-4886-8312-336685769500 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:51:08.071481 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6773997-95e9-4886-8312-336685769500 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:08.072917 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6773997-95e9-4886-8312-336685769500 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:08.099356 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 533/1053] 10.222.0.13 () {56 vars in 1133 bytes} [Tue Jun 7 21:51:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:08.101810 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a73cfe-afaa-4a60-97fc-cb4cc1e74dae None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:08.155140 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:08.211749 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a73cfe-afaa-4a60-97fc-cb4cc1e74dae None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:08.246532 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b0c68-978e-4d24-b3bb-912c257ba7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:08.247663 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b0c68-978e-4d24-b3bb-912c257ba7ed None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:08.248087 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b0c68-978e-4d24-b3bb-912c257ba7ed None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:08.248529 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b0c68-978e-4d24-b3bb-912c257ba7ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:08.259656 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b0c68-978e-4d24-b3bb-912c257ba7ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:08.261313 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b0c68-978e-4d24-b3bb-912c257ba7ed None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:08.263286 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703af096-e638-4d85-b3ae-d958b10f74d7 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:08.263640 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703af096-e638-4d85-b3ae-d958b10f74d7 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:08.263923 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703af096-e638-4d85-b3ae-d958b10f74d7 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:08.265492 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 534/1054] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:51:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 21:51:08.270976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b0bdfb-a6d3-44d1-861c-37986c810138 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:08.271264 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b0bdfb-a6d3-44d1-861c-37986c810138 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:08.271576 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b0bdfb-a6d3-44d1-861c-37986c810138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:08.279685 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-298b0c68-978e-4d24-b3bb-912c257ba7ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:51:08.279685 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:51:08.279685 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:51:08.279685 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:51:08.279685 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:51:08.279685 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:51:08.279685 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:51:08.279685 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:51:08.279685 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:51:08.279685 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:51:08.279685 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:51:08.279685 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:51:08.279685 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:51:08.279685 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:51:08.279685 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:51:08.280766 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:51:08.280766 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:51:08.280766 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:51:08.280766 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:51:08.280766 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:51:08.280766 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:51:08.280766 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:51:08.280766 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:51:08.280766 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:51:08.280766 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:51:08.280766 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:51:08.280766 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:51:08.280766 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:51:08.280766 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:51:08.280766 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:51:08.280766 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:51:08.280766 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:51:08.281935 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:51:08.281935 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:51:08.281935 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:51:08.281935 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:51:08.283868 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 521/1055] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:51:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:51:08.289285 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b0c68-978e-4d24-b3bb-912c257ba7ed None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:08.292161 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-29b0bdfb-a6d3-44d1-861c-37986c810138 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:51:08.293338 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-29b0bdfb-a6d3-44d1-861c-37986c810138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:51:08.321586 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:08.365305 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 535/1056] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:51:08 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 21:51:08.388376 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b0bdfb-a6d3-44d1-861c-37986c810138 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:08.395812 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:08.398733 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b0c68-978e-4d24-b3bb-912c257ba7ed None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:08.435684 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f02189-2e11-4a57-99f1-843081c8add1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:08.436271 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f02189-2e11-4a57-99f1-843081c8add1 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:08.436558 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f02189-2e11-4a57-99f1-843081c8add1 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:08.436834 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f02189-2e11-4a57-99f1-843081c8add1 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:08.437550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f02189-2e11-4a57-99f1-843081c8add1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:51:08.437860 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f02189-2e11-4a57-99f1-843081c8add1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:08.440007 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f02189-2e11-4a57-99f1-843081c8add1 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:08.453665 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 522/1057] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:08.459872 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f02189-2e11-4a57-99f1-843081c8add1 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:08.511535 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b0bdfb-a6d3-44d1-861c-37986c810138 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:08.546778 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f57e992-9221-41ac-b241-7433d3671db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:08.547664 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57e992-9221-41ac-b241-7433d3671db3 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:08.547942 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57e992-9221-41ac-b241-7433d3671db3 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:08.548415 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57e992-9221-41ac-b241-7433d3671db3 None admin] PATH_INFO: `/v3/services/image` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:08.549073 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f57e992-9221-41ac-b241-7433d3671db3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:08.550619 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f57e992-9221-41ac-b241-7433d3671db3 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:08.562454 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-9f57e992-9221-41ac-b241-7433d3671db3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 21:51:08.562454 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:51:08.562454 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 21:51:08.562454 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 21:51:08.562454 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 21:51:08.562454 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 21:51:08.562454 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 21:51:08.562454 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:51:08.562454 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 21:51:08.562454 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:51:08.562454 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 21:51:08.562454 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:51:08.562454 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:51:08.562454 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:51:08.562454 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:51:08.562454 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:51:08.562454 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:51:08.564037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:51:08.564037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:51:08.564037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:51:08.564037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:51:08.564037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:51:08.564037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:51:08.564037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 21:51:08.564037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 21:51:08.564037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 21:51:08.564037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 21:51:08.564037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:51:08.564037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:51:08.564037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:51:08.564037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:51:08.564037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:51:08.564037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:51:08.565412 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:51:08.565412 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:51:08.565412 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:51:08.565412 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:51:08.565412 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:51:08.565412 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:51:08.565412 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:51:08.565412 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:51:08.565412 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:51:08.565412 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:51:08.565412 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 21:51:08.565412 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:51:08.565412 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 21:51:08.565412 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:51:08.565412 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 536/1058] 10.222.0.13 () {56 vars in 1105 bytes} [Tue Jun 7 21:51:08 2022] GET /identity/v3/services/image => generated 86 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:51:08.567402 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f02189-2e11-4a57-99f1-843081c8add1 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:08.570424 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f57e992-9221-41ac-b241-7433d3671db3 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:08.599767 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:08.608444 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc66603-32a1-4937-a50a-9dc1a36bbbb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:08.609244 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc66603-32a1-4937-a50a-9dc1a36bbbb8 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:08.609531 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc66603-32a1-4937-a50a-9dc1a36bbbb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:08.609898 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc66603-32a1-4937-a50a-9dc1a36bbbb8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:08.616913 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc66603-32a1-4937-a50a-9dc1a36bbbb8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:08.618350 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc66603-32a1-4937-a50a-9dc1a36bbbb8 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:08.632832 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-cdc66603-32a1-4937-a50a-9dc1a36bbbb8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:51:08.632832 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:51:08.632832 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:51:08.632832 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:51:08.632832 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:51:08.632832 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:51:08.632832 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:51:08.632832 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:51:08.632832 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:51:08.632832 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:51:08.632832 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:51:08.632832 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:51:08.632832 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 21:51:08.632832 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 21:51:08.632832 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 21:51:08.633833 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 21:51:08.633833 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:51:08.633833 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:51:08.633833 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:51:08.633833 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:51:08.633833 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:51:08.633833 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:51:08.633833 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:51:08.633833 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:51:08.633833 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:51:08.633833 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:51:08.633833 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:51:08.633833 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:51:08.633833 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:51:08.633833 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:51:08.633833 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:51:08.633833 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:51:08.634738 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 21:51:08.634738 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 21:51:08.634738 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 21:51:08.634738 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 21:51:08.634738 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 21:51:08.634738 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 21:51:08.634738 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:51:08.634738 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:51:08.638732 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 523/1059] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:51:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:51:08.639789 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc66603-32a1-4937-a50a-9dc1a36bbbb8 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:08.666873 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:08.676306 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f57e992-9221-41ac-b241-7433d3671db3 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:08.717379 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c681932-9c3c-40f6-b5df-d89e9456e67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:08.717959 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c681932-9c3c-40f6-b5df-d89e9456e67d None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:08.718235 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c681932-9c3c-40f6-b5df-d89e9456e67d None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:08.718510 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c681932-9c3c-40f6-b5df-d89e9456e67d None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:08.719815 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c681932-9c3c-40f6-b5df-d89e9456e67d None admin] RBAC: Adding query filter params (name=image) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:51:08.720253 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c681932-9c3c-40f6-b5df-d89e9456e67d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:08.721811 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c681932-9c3c-40f6-b5df-d89e9456e67d None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:08.726895 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 537/1060] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:51:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:51:08.732432 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c681932-9c3c-40f6-b5df-d89e9456e67d None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:08.761495 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc66603-32a1-4937-a50a-9dc1a36bbbb8 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:08.800263 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880c8894-261a-4e83-b9f3-40dfbfcd7f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:08.801111 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c8894-261a-4e83-b9f3-40dfbfcd7f1c None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:08.801232 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c8894-261a-4e83-b9f3-40dfbfcd7f1c None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:08.801464 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c8894-261a-4e83-b9f3-40dfbfcd7f1c None admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:08.802242 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880c8894-261a-4e83-b9f3-40dfbfcd7f1c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:51:08.802550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880c8894-261a-4e83-b9f3-40dfbfcd7f1c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:08.803998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880c8894-261a-4e83-b9f3-40dfbfcd7f1c None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:08.822016 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 524/1061] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jun 7 21:51:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:51:08.823701 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880c8894-261a-4e83-b9f3-40dfbfcd7f1c None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:08.833271 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:08.845098 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c681932-9c3c-40f6-b5df-d89e9456e67d None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:08.881209 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c95b332-5c34-4734-9910-ebfb4752781e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:08.881874 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95b332-5c34-4734-9910-ebfb4752781e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:08.882154 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95b332-5c34-4734-9910-ebfb4752781e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:08.882429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95b332-5c34-4734-9910-ebfb4752781e None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:08.883130 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c95b332-5c34-4734-9910-ebfb4752781e None admin] RBAC: Adding query filter params (type=image) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:51:08.883579 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c95b332-5c34-4734-9910-ebfb4752781e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:08.884939 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c95b332-5c34-4734-9910-ebfb4752781e None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:08.889942 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 538/1062] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:51:08 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:51:08.895895 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c95b332-5c34-4734-9910-ebfb4752781e None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:08.910420 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:08.932695 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880c8894-261a-4e83-b9f3-40dfbfcd7f1c None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:08.968639 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c2a18-f1a0-42d8-a742-32ff35c1e3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:08.970140 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c2a18-f1a0-42d8-a742-32ff35c1e3d5 None admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:08.970420 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c2a18-f1a0-42d8-a742-32ff35c1e3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:08.970692 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c2a18-f1a0-42d8-a742-32ff35c1e3d5 None admin] PATH_INFO: `/v3/projects/0219dbda54274fdc9bdccbd960c30c6b/users/dbe0303ea0764347bfb054870ebba41f/roles/14b78e4dcfb84633981a9b2f2f06f4fb` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:08.973195 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3c2a18-f1a0-42d8-a742-32ff35c1e3d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=0219dbda54274fdc9bdccbd960c30c6b, user_id=dbe0303ea0764347bfb054870ebba41f, role_id=14b78e4dcfb84633981a9b2f2f06f4fb)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:08.974614 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3c2a18-f1a0-42d8-a742-32ff35c1e3d5 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:08.990294 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 525/1063] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jun 7 21:51:08 2022] PUT /identity/v3/projects/0219dbda54274fdc9bdccbd960c30c6b/users/dbe0303ea0764347bfb054870ebba41f/roles/14b78e4dcfb84633981a9b2f2f06f4fb => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 21:51:09.011633 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c95b332-5c34-4734-9910-ebfb4752781e None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:09.050919 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e03f9de-f62f-41d8-ab7b-695973c722fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:09.051646 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03f9de-f62f-41d8-ab7b-695973c722fe None admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:09.051934 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03f9de-f62f-41d8-ab7b-695973c722fe None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:09.052209 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03f9de-f62f-41d8-ab7b-695973c722fe None admin] PATH_INFO: `/v3/endpoints` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:09.052976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e03f9de-f62f-41d8-ab7b-695973c722fe None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:09.054401 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e03f9de-f62f-41d8-ab7b-695973c722fe None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:09.083364 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 539/1064] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Jun 7 21:51:08 2022] POST /identity/v3/endpoints => generated 327 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 21:51:09.997125 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975bd03c-b5e9-4c20-83ab-f0c84b647019 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:09.997690 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975bd03c-b5e9-4c20-83ab-f0c84b647019 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:09.997690 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975bd03c-b5e9-4c20-83ab-f0c84b647019 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:09.999174 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 526/1065] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:51:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:51:10.004747 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a830c5d-4347-4c34-a540-e15c5f45acca None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:10.005046 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a830c5d-4347-4c34-a540-e15c5f45acca None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:10.005331 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a830c5d-4347-4c34-a540-e15c5f45acca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:10.022029 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-5a830c5d-4347-4c34-a540-e15c5f45acca None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:51:10.023152 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-5a830c5d-4347-4c34-a540-e15c5f45acca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:51:10.031110 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:10.100611 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 540/1066] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:51:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:51:10.124681 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975bd03c-b5e9-4c20-83ab-f0c84b647019 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:10.233650 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975bd03c-b5e9-4c20-83ab-f0c84b647019 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:10.269699 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec15b51b-4f39-4a77-bb75-f6bb7279c965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:10.270493 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec15b51b-4f39-4a77-bb75-f6bb7279c965 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:10.270776 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec15b51b-4f39-4a77-bb75-f6bb7279c965 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:10.271059 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec15b51b-4f39-4a77-bb75-f6bb7279c965 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:10.278985 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:10.284004 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec15b51b-4f39-4a77-bb75-f6bb7279c965 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:51:10.292709 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-ec15b51b-4f39-4a77-bb75-f6bb7279c965 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:51:10.292709 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:51:10.292709 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:51:10.292709 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:51:10.292709 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:51:10.292709 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:51:10.292709 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:51:10.292709 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:51:10.292709 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:51:10.292709 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:51:10.292709 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:51:10.292709 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:51:10.292709 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:51:10.292709 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:51:10.292709 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:51:10.292709 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:51:10.293748 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:51:10.293748 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:51:10.293748 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:51:10.293748 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:51:10.293748 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:51:10.293748 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:51:10.293748 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:51:10.293748 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:51:10.293748 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:51:10.293748 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:51:10.293748 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:51:10.293748 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:51:10.293748 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:51:10.293748 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:51:10.293748 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:51:10.293748 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:51:10.295254 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:51:10.295254 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:51:10.295254 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:51:10.295254 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:51:10.295254 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:51:10.295254 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:51:10.295254 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:51:10.295254 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:51:10.295254 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:51:10.295254 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:51:10.295254 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:51:10.295254 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:51:10.295254 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 527/1067] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jun 7 21:51:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:51:10.300856 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a830c5d-4347-4c34-a540-e15c5f45acca None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:10.401593 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a830c5d-4347-4c34-a540-e15c5f45acca None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:10.435522 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d8f37-e5f0-4fa9-a5e6-d0f2f7f8013b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:10.436185 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d8f37-e5f0-4fa9-a5e6-d0f2f7f8013b None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:10.436467 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d8f37-e5f0-4fa9-a5e6-d0f2f7f8013b None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:10.436743 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d8f37-e5f0-4fa9-a5e6-d0f2f7f8013b None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:10.437480 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7d8f37-e5f0-4fa9-a5e6-d0f2f7f8013b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:51:10.437816 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7d8f37-e5f0-4fa9-a5e6-d0f2f7f8013b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:10.439297 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7d8f37-e5f0-4fa9-a5e6-d0f2f7f8013b None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:10.444220 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:10.448401 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 541/1068] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:51:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:10.454471 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec15b51b-4f39-4a77-bb75-f6bb7279c965 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:10.556071 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec15b51b-4f39-4a77-bb75-f6bb7279c965 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:10.589293 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a45e52b-d940-4c6c-90b8-359d28b7faaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:10.590162 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45e52b-d940-4c6c-90b8-359d28b7faaa None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:10.590443 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45e52b-d940-4c6c-90b8-359d28b7faaa None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:10.590717 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45e52b-d940-4c6c-90b8-359d28b7faaa None admin] PATH_INFO: `/v3/users/4a284674692942e69800e67b57bbdaf2` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:10.604928 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a45e52b-d940-4c6c-90b8-359d28b7faaa None admin] RBAC: Authorizing `identity:get_user(user_id=4a284674692942e69800e67b57bbdaf2)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:10.606364 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a45e52b-d940-4c6c-90b8-359d28b7faaa None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:10.608228 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 528/1069] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:51:10 2022] GET /identity/v3/users/4a284674692942e69800e67b57bbdaf2 => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:51:10.613826 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d8f37-e5f0-4fa9-a5e6-d0f2f7f8013b None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:10.712905 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d8f37-e5f0-4fa9-a5e6-d0f2f7f8013b None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:10.748509 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b20c1b-f9b4-415f-b584-b820d2d12926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:10.749161 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b20c1b-f9b4-415f-b584-b820d2d12926 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:10.749442 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b20c1b-f9b4-415f-b584-b820d2d12926 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:10.749716 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b20c1b-f9b4-415f-b584-b820d2d12926 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:10.822547 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:10.826062 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b20c1b-f9b4-415f-b584-b820d2d12926 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:10.827689 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b20c1b-f9b4-415f-b584-b820d2d12926 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:10.928406 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 542/1070] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:51:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 21:51:10.934065 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a45e52b-d940-4c6c-90b8-359d28b7faaa None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:10.998167 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:11.042239 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a45e52b-d940-4c6c-90b8-359d28b7faaa None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:11.080575 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d0f516-8ec8-4ef8-b095-8cb1ca971e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:11.081372 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d0f516-8ec8-4ef8-b095-8cb1ca971e68 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:11.081655 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d0f516-8ec8-4ef8-b095-8cb1ca971e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:11.081931 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d0f516-8ec8-4ef8-b095-8cb1ca971e68 None admin] PATH_INFO: `/v3/projects/938e609f0dbd4b54b509fd8c09d15796` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:11.083163 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d0f516-8ec8-4ef8-b095-8cb1ca971e68 None admin] RBAC: Authorizing `identity:get_project(project_id=938e609f0dbd4b54b509fd8c09d15796)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:11.084664 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d0f516-8ec8-4ef8-b095-8cb1ca971e68 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:11.086565 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 529/1071] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:51:10 2022] GET /identity/v3/projects/938e609f0dbd4b54b509fd8c09d15796 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:11.092395 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b20c1b-f9b4-415f-b584-b820d2d12926 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:11.192303 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:11.196357 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b20c1b-f9b4-415f-b584-b820d2d12926 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:11.231809 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5767852b-4fa3-43d8-a050-9e879ee0eafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:11.232468 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767852b-4fa3-43d8-a050-9e879ee0eafe None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:11.232749 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767852b-4fa3-43d8-a050-9e879ee0eafe None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:11.233022 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767852b-4fa3-43d8-a050-9e879ee0eafe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:11.233846 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5767852b-4fa3-43d8-a050-9e879ee0eafe None admin] RBAC: Adding query filter params (role.id=6b483758dbd44eb4a78df2ce4603d601, scope.project.id=938e609f0dbd4b54b509fd8c09d15796, user.id=4a284674692942e69800e67b57bbdaf2) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:51:11.234158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5767852b-4fa3-43d8-a050-9e879ee0eafe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:11.235747 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5767852b-4fa3-43d8-a050-9e879ee0eafe None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:11.247374 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 543/1072] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:51:11 2022] GET /identity/v3/role_assignments?user.id=4a284674692942e69800e67b57bbdaf2&scope.project.id=938e609f0dbd4b54b509fd8c09d15796&role.id=6b483758dbd44eb4a78df2ce4603d601 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:11.287168 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c5524-41e2-4a93-b45f-7ced6485a410 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:11.287851 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c5524-41e2-4a93-b45f-7ced6485a410 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:11.288372 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c5524-41e2-4a93-b45f-7ced6485a410 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:11.290388 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 530/1073] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:51:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:51:11.296451 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945f18d-9870-4863-a548-ae3c58264290 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:11.297079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945f18d-9870-4863-a548-ae3c58264290 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:11.298203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945f18d-9870-4863-a548-ae3c58264290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:11.316943 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-3945f18d-9870-4863-a548-ae3c58264290 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:51:11.318177 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-3945f18d-9870-4863-a548-ae3c58264290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:51:11.393008 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 544/1074] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:51:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:51:11.400317 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e8912-c1f7-44be-b51e-9b2386b103a2 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:11.400604 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e8912-c1f7-44be-b51e-9b2386b103a2 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:11.401066 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e8912-c1f7-44be-b51e-9b2386b103a2 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:11.402440 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 531/1075] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:51:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:51:11.408355 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f22ea-7ddb-43dd-a201-70af79d10d39 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:11.408655 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f22ea-7ddb-43dd-a201-70af79d10d39 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:11.408931 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f22ea-7ddb-43dd-a201-70af79d10d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:11.416619 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e8912-c1f7-44be-b51e-9b2386b103a2 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:11.426941 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-2e5f22ea-7ddb-43dd-a201-70af79d10d39 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:51:11.428193 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-2e5f22ea-7ddb-43dd-a201-70af79d10d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:51:11.507368 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 545/1076] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:51:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:51:11.529335 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5f22ea-7ddb-43dd-a201-70af79d10d39 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:11.539313 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e8912-c1f7-44be-b51e-9b2386b103a2 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:11.557213 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:11.567702 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:11.577792 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92706819-8fba-43fd-9f8f-a7fbefe59e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:11.578638 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92706819-8fba-43fd-9f8f-a7fbefe59e08 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:11.579470 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92706819-8fba-43fd-9f8f-a7fbefe59e08 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:11.579749 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92706819-8fba-43fd-9f8f-a7fbefe59e08 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:11.590711 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92706819-8fba-43fd-9f8f-a7fbefe59e08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:51:11.599759 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-92706819-8fba-43fd-9f8f-a7fbefe59e08 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 21:51:11.599759 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:51:11.599759 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:51:11.599759 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:51:11.599759 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:51:11.599759 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:51:11.599759 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:51:11.599759 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:51:11.599759 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:51:11.599759 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:51:11.599759 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:51:11.599759 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:51:11.599759 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:51:11.599759 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:51:11.599759 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:51:11.599759 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:51:11.600716 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:51:11.600716 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:51:11.600716 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:51:11.600716 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:51:11.600716 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:51:11.600716 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:51:11.600716 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:51:11.600716 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:51:11.600716 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:51:11.600716 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:51:11.600716 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:51:11.600716 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:51:11.600716 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:51:11.600716 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:51:11.600716 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:51:11.600716 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:51:11.601576 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:51:11.601576 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:51:11.601576 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:51:11.601576 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:51:11.601576 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:51:11.601576 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:51:11.601576 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:51:11.601576 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:51:11.601576 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:51:11.601576 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:51:11.601576 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 21:51:11.601576 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:51:11.601576 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 532/1077] 10.222.0.13 () {56 vars in 1120 bytes} [Tue Jun 7 21:51:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:51:11.606972 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92706819-8fba-43fd-9f8f-a7fbefe59e08 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:11.642527 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5f22ea-7ddb-43dd-a201-70af79d10d39 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:11.680867 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2daf73e-c827-4dd1-88bf-606f0487ec15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:11.681446 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2daf73e-c827-4dd1-88bf-606f0487ec15 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:11.681737 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2daf73e-c827-4dd1-88bf-606f0487ec15 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:11.682011 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2daf73e-c827-4dd1-88bf-606f0487ec15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:11.717346 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92706819-8fba-43fd-9f8f-a7fbefe59e08 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:11.754077 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2daf73e-c827-4dd1-88bf-606f0487ec15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:11.754761 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2902188d-92e2-4676-b676-cb7caeb24c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:11.755558 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2902188d-92e2-4676-b676-cb7caeb24c96 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:11.755919 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2902188d-92e2-4676-b676-cb7caeb24c96 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:11.756270 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2902188d-92e2-4676-b676-cb7caeb24c96 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:11.756779 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2daf73e-c827-4dd1-88bf-606f0487ec15 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:11.757169 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2902188d-92e2-4676-b676-cb7caeb24c96 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:51:11.757547 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2902188d-92e2-4676-b676-cb7caeb24c96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:11.759009 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2902188d-92e2-4676-b676-cb7caeb24c96 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:11.775546 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 533/1078] 10.222.0.13 () {56 vars in 1115 bytes} [Tue Jun 7 21:51:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:51:11.776924 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2902188d-92e2-4676-b676-cb7caeb24c96 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:11.782993 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:11.833540 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:11.863798 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 546/1079] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:51:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 07 21:51:11.866628 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2daf73e-c827-4dd1-88bf-606f0487ec15 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:11.889361 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2902188d-92e2-4676-b676-cb7caeb24c96 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:11.927680 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79bd635-fc0b-4289-a5ef-ed61bcf996be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:11.928730 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79bd635-fc0b-4289-a5ef-ed61bcf996be None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:11.929019 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79bd635-fc0b-4289-a5ef-ed61bcf996be None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:11.929293 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79bd635-fc0b-4289-a5ef-ed61bcf996be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:11.941298 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79bd635-fc0b-4289-a5ef-ed61bcf996be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:11.942684 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79bd635-fc0b-4289-a5ef-ed61bcf996be None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:11.960139 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-f79bd635-fc0b-4289-a5ef-ed61bcf996be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:51:11.960139 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:51:11.960139 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:51:11.960139 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:51:11.960139 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:51:11.960139 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:51:11.960139 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:51:11.960139 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:51:11.960139 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:51:11.960139 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:51:11.960139 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:51:11.960139 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:51:11.960139 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:51:11.960139 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:51:11.960139 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:51:11.961079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:51:11.961079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:51:11.961079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:51:11.961079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:51:11.961079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:51:11.961079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:51:11.961079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:51:11.961079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:51:11.961079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:51:11.961079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:51:11.961079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:51:11.961079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:51:11.961079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:51:11.961079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:51:11.961079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:51:11.961079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:51:11.961079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:51:11.961980 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:51:11.961980 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:51:11.961980 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:51:11.961980 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:51:11.966062 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 534/1080] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:51:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:51:11.966916 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79bd635-fc0b-4289-a5ef-ed61bcf996be None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:11.975100 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2daf73e-c827-4dd1-88bf-606f0487ec15 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:12.004054 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:12.018138 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4948a80-8ff3-4847-8094-d465256539db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:12.018921 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4948a80-8ff3-4847-8094-d465256539db None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:12.019206 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4948a80-8ff3-4847-8094-d465256539db None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:12.019745 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4948a80-8ff3-4847-8094-d465256539db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:12.030319 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4948a80-8ff3-4847-8094-d465256539db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:12.031872 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4948a80-8ff3-4847-8094-d465256539db None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:12.053662 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-e4948a80-8ff3-4847-8094-d465256539db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:51:12.053662 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:51:12.053662 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:51:12.053662 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:51:12.053662 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:51:12.053662 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:51:12.053662 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:51:12.053662 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:51:12.053662 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:51:12.053662 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:51:12.053662 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:51:12.053662 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:51:12.053662 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:51:12.053662 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:51:12.053662 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:51:12.054697 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:51:12.054697 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:51:12.054697 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:51:12.054697 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:51:12.054697 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:51:12.054697 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:51:12.054697 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:51:12.054697 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:51:12.054697 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:51:12.054697 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:51:12.054697 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:51:12.054697 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:51:12.054697 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:51:12.054697 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:51:12.054697 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:51:12.054697 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:51:12.054697 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:51:12.055662 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:51:12.055662 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:51:12.055662 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:51:12.055662 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:51:12.059466 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 547/1081] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:51:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:51:12.061503 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4948a80-8ff3-4847-8094-d465256539db None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:12.078032 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:12.082214 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79bd635-fc0b-4289-a5ef-ed61bcf996be None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:12.121910 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff297a-7ccf-410e-985c-32388617b796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:12.122570 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff297a-7ccf-410e-985c-32388617b796 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:12.122857 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff297a-7ccf-410e-985c-32388617b796 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:12.123141 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff297a-7ccf-410e-985c-32388617b796 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:12.123889 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aff297a-7ccf-410e-985c-32388617b796 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:51:12.124206 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aff297a-7ccf-410e-985c-32388617b796 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:12.126242 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aff297a-7ccf-410e-985c-32388617b796 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:12.139019 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 535/1082] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:51:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:12.145060 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff297a-7ccf-410e-985c-32388617b796 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:12.166006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4948a80-8ff3-4847-8094-d465256539db None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:12.201232 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baad2b3-0196-462f-b3d3-95fb5e6cb5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:12.201832 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baad2b3-0196-462f-b3d3-95fb5e6cb5e0 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:12.202121 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baad2b3-0196-462f-b3d3-95fb5e6cb5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:12.202402 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baad2b3-0196-462f-b3d3-95fb5e6cb5e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:12.203159 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baad2b3-0196-462f-b3d3-95fb5e6cb5e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:51:12.203629 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baad2b3-0196-462f-b3d3-95fb5e6cb5e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:12.205101 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baad2b3-0196-462f-b3d3-95fb5e6cb5e0 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:12.216599 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 548/1083] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:51:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:51:12.267893 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff297a-7ccf-410e-985c-32388617b796 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:12.307536 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc155d7-71bc-42f4-80f5-a5a4ab278881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:12.308341 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc155d7-71bc-42f4-80f5-a5a4ab278881 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:12.308621 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc155d7-71bc-42f4-80f5-a5a4ab278881 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:12.308895 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc155d7-71bc-42f4-80f5-a5a4ab278881 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:12.317104 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc155d7-71bc-42f4-80f5-a5a4ab278881 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:12.318552 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc155d7-71bc-42f4-80f5-a5a4ab278881 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:12.333844 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-3fc155d7-71bc-42f4-80f5-a5a4ab278881 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 21:51:12.333844 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:51:12.333844 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:51:12.333844 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:51:12.333844 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:51:12.333844 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:51:12.333844 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:51:12.333844 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:51:12.333844 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:51:12.333844 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:51:12.333844 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:51:12.333844 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:51:12.333844 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 21:51:12.333844 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 21:51:12.333844 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 21:51:12.333844 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 21:51:12.334906 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:51:12.334906 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:51:12.334906 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 21:51:12.334906 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:51:12.334906 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 21:51:12.334906 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:51:12.334906 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:51:12.334906 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:51:12.334906 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:51:12.334906 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:51:12.334906 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:51:12.334906 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:51:12.334906 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:51:12.334906 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:51:12.334906 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:51:12.334906 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:51:12.334906 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:51:12.334906 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:51:12.336541 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:51:12.336541 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:51:12.336541 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 21:51:12.336541 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 21:51:12.336541 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 21:51:12.336541 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 21:51:12.336541 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 21:51:12.336541 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 21:51:12.336541 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 21:51:12.336541 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:51:12.336541 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 536/1084] 10.222.0.13 () {56 vars in 1093 bytes} [Tue Jun 7 21:51:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:51:12.341045 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baad2b3-0196-462f-b3d3-95fb5e6cb5e0 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:12.384472 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:12.456442 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baad2b3-0196-462f-b3d3-95fb5e6cb5e0 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:12.491864 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb8e9d-5652-47a7-a675-b75c1df99133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:12.492549 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb8e9d-5652-47a7-a675-b75c1df99133 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:12.492875 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb8e9d-5652-47a7-a675-b75c1df99133 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:12.493111 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb8e9d-5652-47a7-a675-b75c1df99133 None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:12.493947 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beb8e9d-5652-47a7-a675-b75c1df99133 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:51:12.494259 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beb8e9d-5652-47a7-a675-b75c1df99133 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:12.495723 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beb8e9d-5652-47a7-a675-b75c1df99133 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:12.515431 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 549/1085] 10.222.0.13 () {56 vars in 1133 bytes} [Tue Jun 7 21:51:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:12.522084 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc155d7-71bc-42f4-80f5-a5a4ab278881 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:12.546628 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:12.629458 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc155d7-71bc-42f4-80f5-a5a4ab278881 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:12.662914 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a4c56-946c-4a57-b3d2-17bccb81a065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:12.663589 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a4c56-946c-4a57-b3d2-17bccb81a065 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:12.663879 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a4c56-946c-4a57-b3d2-17bccb81a065 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:12.664160 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a4c56-946c-4a57-b3d2-17bccb81a065 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:12.729388 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3a4c56-946c-4a57-b3d2-17bccb81a065 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:12.730757 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3a4c56-946c-4a57-b3d2-17bccb81a065 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:12.751260 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:12.834737 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 537/1086] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:51:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 21:51:12.840405 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb8e9d-5652-47a7-a675-b75c1df99133 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:12.932768 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:12.940677 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb8e9d-5652-47a7-a675-b75c1df99133 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:12.974098 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfcd92d-2615-4c54-9064-e712511bdd4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:12.974887 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfcd92d-2615-4c54-9064-e712511bdd4b None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:12.975167 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfcd92d-2615-4c54-9064-e712511bdd4b None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:12.975474 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfcd92d-2615-4c54-9064-e712511bdd4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:12.985919 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfcd92d-2615-4c54-9064-e712511bdd4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:12.987297 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfcd92d-2615-4c54-9064-e712511bdd4b None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:13.005070 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-cbfcd92d-2615-4c54-9064-e712511bdd4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:51:13.005070 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:51:13.005070 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:51:13.005070 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:51:13.005070 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:51:13.005070 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:51:13.005070 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:51:13.005070 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:51:13.005070 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:51:13.005070 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:51:13.005070 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:51:13.005070 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:51:13.005070 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:51:13.005070 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:51:13.005070 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:51:13.006069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:51:13.006069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:51:13.006069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:51:13.006069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:51:13.006069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:51:13.006069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:51:13.006069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:51:13.006069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:51:13.006069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:51:13.006069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:51:13.006069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:51:13.006069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:51:13.006069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:51:13.006069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:51:13.006069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:51:13.006069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:51:13.006069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:51:13.007414 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:51:13.007414 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:51:13.007414 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:51:13.007414 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:51:13.007414 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 550/1087] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:51:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:51:13.011907 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a4c56-946c-4a57-b3d2-17bccb81a065 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:13.114674 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a4c56-946c-4a57-b3d2-17bccb81a065 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:13.134479 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:13.148666 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60436833-3829-4307-a834-df7d5e2c464d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:13.149323 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60436833-3829-4307-a834-df7d5e2c464d None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:13.149615 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60436833-3829-4307-a834-df7d5e2c464d None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:13.149880 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60436833-3829-4307-a834-df7d5e2c464d None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:13.150598 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60436833-3829-4307-a834-df7d5e2c464d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:51:13.150903 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60436833-3829-4307-a834-df7d5e2c464d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:13.152294 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60436833-3829-4307-a834-df7d5e2c464d None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:13.164025 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 538/1088] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:51:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:13.169641 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfcd92d-2615-4c54-9064-e712511bdd4b None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:13.269919 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfcd92d-2615-4c54-9064-e712511bdd4b None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:13.303728 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b63c81-743f-4bdc-8f5e-c83ffc38b7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:13.304515 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b63c81-743f-4bdc-8f5e-c83ffc38b7c2 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:13.304798 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b63c81-743f-4bdc-8f5e-c83ffc38b7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:13.305075 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b63c81-743f-4bdc-8f5e-c83ffc38b7c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:13.311647 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b63c81-743f-4bdc-8f5e-c83ffc38b7c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:13.313263 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b63c81-743f-4bdc-8f5e-c83ffc38b7c2 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:13.326749 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-e5b63c81-743f-4bdc-8f5e-c83ffc38b7c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:51:13.326749 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:51:13.326749 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:51:13.326749 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:51:13.326749 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:51:13.326749 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:51:13.326749 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:51:13.326749 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:51:13.326749 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:51:13.326749 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:51:13.326749 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:51:13.326749 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:51:13.326749 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 21:51:13.326749 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 21:51:13.326749 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 21:51:13.327886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 21:51:13.327886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:51:13.327886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:51:13.327886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:51:13.327886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:51:13.327886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:51:13.327886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:51:13.327886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:51:13.327886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:51:13.327886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:51:13.327886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:51:13.327886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:51:13.327886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:51:13.327886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:51:13.327886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:51:13.327886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:51:13.327886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:51:13.329247 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 21:51:13.329247 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 21:51:13.329247 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 21:51:13.329247 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 21:51:13.329247 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 21:51:13.329247 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 21:51:13.329247 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:51:13.329247 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:51:13.331483 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 551/1089] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:51:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:51:13.333767 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60436833-3829-4307-a834-df7d5e2c464d None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:13.432028 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60436833-3829-4307-a834-df7d5e2c464d None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:13.465307 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528d12ca-625d-4547-8a52-f3a705bcec35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:13.465971 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d12ca-625d-4547-8a52-f3a705bcec35 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:13.466259 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d12ca-625d-4547-8a52-f3a705bcec35 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:13.466538 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d12ca-625d-4547-8a52-f3a705bcec35 None admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:13.467360 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528d12ca-625d-4547-8a52-f3a705bcec35 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:51:13.467663 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528d12ca-625d-4547-8a52-f3a705bcec35 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:13.469760 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528d12ca-625d-4547-8a52-f3a705bcec35 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:13.481917 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 539/1090] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jun 7 21:51:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:13.488078 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b63c81-743f-4bdc-8f5e-c83ffc38b7c2 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:13.497688 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:13.511361 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13babd86-3dac-476d-a76a-90fd3298afa8 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:13.511654 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13babd86-3dac-476d-a76a-90fd3298afa8 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:13.511934 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13babd86-3dac-476d-a76a-90fd3298afa8 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:13.513300 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 540/1091] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:51:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 21:51:13.518739 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb82ca-a617-45f8-b8a4-5e57bf21e470 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:13.519023 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb82ca-a617-45f8-b8a4-5e57bf21e470 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:13.519297 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb82ca-a617-45f8-b8a4-5e57bf21e470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:13.536510 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-4adb82ca-a617-45f8-b8a4-5e57bf21e470 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:51:13.537708 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-4adb82ca-a617-45f8-b8a4-5e57bf21e470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:51:13.571413 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:13.590675 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b63c81-743f-4bdc-8f5e-c83ffc38b7c2 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:13.605397 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 541/1092] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:51:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 21:51:13.623881 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9443c25d-c9c2-46be-98aa-6e186fff5ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:13.624710 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443c25d-c9c2-46be-98aa-6e186fff5ab0 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:13.625057 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443c25d-c9c2-46be-98aa-6e186fff5ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:13.625375 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443c25d-c9c2-46be-98aa-6e186fff5ab0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:13.626689 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9443c25d-c9c2-46be-98aa-6e186fff5ab0 None admin] RBAC: Adding query filter params (role.id=14b78e4dcfb84633981a9b2f2f06f4fb, scope.project.id=0219dbda54274fdc9bdccbd960c30c6b, user.id=dbe0303ea0764347bfb054870ebba41f) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:51:13.627074 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9443c25d-c9c2-46be-98aa-6e186fff5ab0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:13.628552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9443c25d-c9c2-46be-98aa-6e186fff5ab0 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:13.629208 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb82ca-a617-45f8-b8a4-5e57bf21e470 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:13.634586 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 552/1093] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:51:13 2022] GET /identity/v3/role_assignments?user.id=dbe0303ea0764347bfb054870ebba41f&scope.project.id=0219dbda54274fdc9bdccbd960c30c6b&role.id=14b78e4dcfb84633981a9b2f2f06f4fb => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:51:13.735393 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb82ca-a617-45f8-b8a4-5e57bf21e470 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:13.775734 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a65ce43-e57c-4450-af9f-13e1f71f1bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:13.776552 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65ce43-e57c-4450-af9f-13e1f71f1bdc None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:13.776837 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65ce43-e57c-4450-af9f-13e1f71f1bdc None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:13.777116 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65ce43-e57c-4450-af9f-13e1f71f1bdc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:13.789189 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a65ce43-e57c-4450-af9f-13e1f71f1bdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:51:13.796870 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-2a65ce43-e57c-4450-af9f-13e1f71f1bdc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:51:13.796870 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:51:13.796870 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:51:13.796870 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:51:13.796870 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:51:13.796870 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:51:13.796870 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:51:13.796870 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:51:13.796870 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:51:13.796870 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:51:13.796870 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:51:13.796870 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:51:13.796870 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:51:13.796870 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:51:13.796870 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:51:13.796870 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:51:13.798070 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:51:13.798070 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:51:13.798070 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:51:13.798070 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:51:13.798070 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:51:13.798070 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:51:13.798070 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:51:13.798070 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:51:13.798070 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:51:13.798070 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:51:13.798070 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:51:13.798070 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:51:13.798070 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:51:13.798070 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:51:13.798070 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:51:13.798070 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:51:13.799224 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:51:13.799224 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:51:13.799224 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:51:13.799224 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:51:13.799224 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:51:13.799224 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:51:13.799224 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:51:13.799224 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:51:13.799224 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:51:13.799224 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:51:13.799224 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:51:13.799224 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:51:13.799224 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 542/1094] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jun 7 21:51:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:51:13.804662 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9443c25d-c9c2-46be-98aa-6e186fff5ab0 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:13.888086 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:13.927750 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9443c25d-c9c2-46be-98aa-6e186fff5ab0 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:13.962058 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d45060-dcc7-4402-9a36-fc4a66e4cfb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:13.962731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d45060-dcc7-4402-9a36-fc4a66e4cfb8 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:13.963011 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d45060-dcc7-4402-9a36-fc4a66e4cfb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:13.963286 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d45060-dcc7-4402-9a36-fc4a66e4cfb8 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:13.964064 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d45060-dcc7-4402-9a36-fc4a66e4cfb8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:51:13.964366 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d45060-dcc7-4402-9a36-fc4a66e4cfb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:13.965725 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d45060-dcc7-4402-9a36-fc4a66e4cfb8 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:13.974334 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 553/1095] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:51:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:13.980480 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a65ce43-e57c-4450-af9f-13e1f71f1bdc None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:14.018095 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:14.088058 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a65ce43-e57c-4450-af9f-13e1f71f1bdc None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:14.123244 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b54c0b-c08b-43ac-8631-4404bb15a412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:14.124139 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b54c0b-c08b-43ac-8631-4404bb15a412 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:14.124418 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b54c0b-c08b-43ac-8631-4404bb15a412 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:14.124699 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b54c0b-c08b-43ac-8631-4404bb15a412 None admin] PATH_INFO: `/v3/users/4a284674692942e69800e67b57bbdaf2` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:14.126049 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b54c0b-c08b-43ac-8631-4404bb15a412 None admin] RBAC: Authorizing `identity:get_user(user_id=4a284674692942e69800e67b57bbdaf2)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:14.127499 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b54c0b-c08b-43ac-8631-4404bb15a412 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:14.129119 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 543/1096] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:51:13 2022] GET /identity/v3/users/4a284674692942e69800e67b57bbdaf2 => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:14.135172 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d45060-dcc7-4402-9a36-fc4a66e4cfb8 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:14.250364 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d45060-dcc7-4402-9a36-fc4a66e4cfb8 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:14.279622 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:14.285804 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e20ed8d-c3cf-4129-97d9-082bb1910cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:14.286840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e20ed8d-c3cf-4129-97d9-082bb1910cf1 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:14.287119 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e20ed8d-c3cf-4129-97d9-082bb1910cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:14.287419 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e20ed8d-c3cf-4129-97d9-082bb1910cf1 None admin] PATH_INFO: `/v3/projects/938e609f0dbd4b54b509fd8c09d15796` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:14.288592 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e20ed8d-c3cf-4129-97d9-082bb1910cf1 None admin] RBAC: Authorizing `identity:get_project(project_id=938e609f0dbd4b54b509fd8c09d15796)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:14.290021 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e20ed8d-c3cf-4129-97d9-082bb1910cf1 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:14.292098 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 554/1097] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:51:14 2022] GET /identity/v3/projects/938e609f0dbd4b54b509fd8c09d15796 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:14.297910 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b54c0b-c08b-43ac-8631-4404bb15a412 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:14.397440 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:14.401418 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b54c0b-c08b-43ac-8631-4404bb15a412 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:14.436140 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba20c0d-306c-4824-b420-27cdd6f6d4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:14.437032 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba20c0d-306c-4824-b420-27cdd6f6d4e7 None admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:14.437311 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba20c0d-306c-4824-b420-27cdd6f6d4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:14.437588 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba20c0d-306c-4824-b420-27cdd6f6d4e7 None admin] PATH_INFO: `/v3/projects/938e609f0dbd4b54b509fd8c09d15796/users/4a284674692942e69800e67b57bbdaf2/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:14.439642 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba20c0d-306c-4824-b420-27cdd6f6d4e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=938e609f0dbd4b54b509fd8c09d15796, user_id=4a284674692942e69800e67b57bbdaf2, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:14.441066 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba20c0d-306c-4824-b420-27cdd6f6d4e7 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:14.454793 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 544/1098] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jun 7 21:51:14 2022] PUT /identity/v3/projects/938e609f0dbd4b54b509fd8c09d15796/users/4a284674692942e69800e67b57bbdaf2/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 21:51:16.750639 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc387ce5-42d3-4f02-a5ec-fb52a0af0b90 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:16.751174 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc387ce5-42d3-4f02-a5ec-fb52a0af0b90 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:16.751174 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc387ce5-42d3-4f02-a5ec-fb52a0af0b90 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:16.752567 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 555/1099] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:51:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:51:16.758150 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226373a-c697-4e48-bc1f-0702c77633f1 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:16.758511 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226373a-c697-4e48-bc1f-0702c77633f1 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:16.758673 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226373a-c697-4e48-bc1f-0702c77633f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:16.776412 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-f226373a-c697-4e48-bc1f-0702c77633f1 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:51:16.777610 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-f226373a-c697-4e48-bc1f-0702c77633f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:51:16.844315 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 545/1100] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:51:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:51:16.868869 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc387ce5-42d3-4f02-a5ec-fb52a0af0b90 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:16.972939 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc387ce5-42d3-4f02-a5ec-fb52a0af0b90 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:17.005747 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df549d-daa7-4d1b-a41e-a5364f2e2d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:17.006456 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df549d-daa7-4d1b-a41e-a5364f2e2d15 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:17.006734 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df549d-daa7-4d1b-a41e-a5364f2e2d15 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:17.007007 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df549d-daa7-4d1b-a41e-a5364f2e2d15 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:17.017353 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60df549d-daa7-4d1b-a41e-a5364f2e2d15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:51:17.024871 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-60df549d-daa7-4d1b-a41e-a5364f2e2d15 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:51:17.024871 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:51:17.024871 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:51:17.024871 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:51:17.024871 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:51:17.024871 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:51:17.024871 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:51:17.024871 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:51:17.024871 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:51:17.024871 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:51:17.024871 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:51:17.024871 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:51:17.024871 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:51:17.024871 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:51:17.024871 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:51:17.024871 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:51:17.025986 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:51:17.025986 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:51:17.025986 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:51:17.025986 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:51:17.025986 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:51:17.025986 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:51:17.025986 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:51:17.025986 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:51:17.025986 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:51:17.025986 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:51:17.025986 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:51:17.025986 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:51:17.025986 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:51:17.025986 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:51:17.025986 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:51:17.025986 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:51:17.027171 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:51:17.027171 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:51:17.027171 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:51:17.027171 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:51:17.027171 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:51:17.027171 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:51:17.027171 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:51:17.027171 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:51:17.027171 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:51:17.027171 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:51:17.027171 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:51:17.027171 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:51:17.027171 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 556/1101] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jun 7 21:51:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:51:17.032290 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f226373a-c697-4e48-bc1f-0702c77633f1 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:17.125283 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:17.137814 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f226373a-c697-4e48-bc1f-0702c77633f1 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:17.171585 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3781ed3-4668-4a13-aafd-8bdcd47ddcf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:17.172354 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3781ed3-4668-4a13-aafd-8bdcd47ddcf0 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:17.172637 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3781ed3-4668-4a13-aafd-8bdcd47ddcf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:17.172915 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3781ed3-4668-4a13-aafd-8bdcd47ddcf0 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:17.173664 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3781ed3-4668-4a13-aafd-8bdcd47ddcf0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:51:17.173973 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3781ed3-4668-4a13-aafd-8bdcd47ddcf0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:17.175368 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3781ed3-4668-4a13-aafd-8bdcd47ddcf0 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:17.183347 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 546/1102] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:51:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:17.189141 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df549d-daa7-4d1b-a41e-a5364f2e2d15 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:17.246708 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:17.287022 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df549d-daa7-4d1b-a41e-a5364f2e2d15 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:17.319102 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80b5562-b91a-4130-bf15-92d01cff5b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:17.319976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b5562-b91a-4130-bf15-92d01cff5b24 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:17.320265 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b5562-b91a-4130-bf15-92d01cff5b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:17.320539 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b5562-b91a-4130-bf15-92d01cff5b24 None admin] PATH_INFO: `/v3/users/4a284674692942e69800e67b57bbdaf2` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:17.321701 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80b5562-b91a-4130-bf15-92d01cff5b24 None admin] RBAC: Authorizing `identity:get_user(user_id=4a284674692942e69800e67b57bbdaf2)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:17.323121 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80b5562-b91a-4130-bf15-92d01cff5b24 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:17.324738 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 557/1103] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:51:17 2022] GET /identity/v3/users/4a284674692942e69800e67b57bbdaf2 => generated 285 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:17.331131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3781ed3-4668-4a13-aafd-8bdcd47ddcf0 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:17.428651 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3781ed3-4668-4a13-aafd-8bdcd47ddcf0 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:17.460944 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d302bf-322e-4d37-a434-fd096f5f4d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:17.461576 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d302bf-322e-4d37-a434-fd096f5f4d42 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:17.461856 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d302bf-322e-4d37-a434-fd096f5f4d42 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:17.462130 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d302bf-322e-4d37-a434-fd096f5f4d42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:17.481226 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:17.529416 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d302bf-322e-4d37-a434-fd096f5f4d42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:17.530834 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d302bf-322e-4d37-a434-fd096f5f4d42 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:17.627200 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 547/1104] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:51:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 21:51:17.633020 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80b5562-b91a-4130-bf15-92d01cff5b24 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:17.728484 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80b5562-b91a-4130-bf15-92d01cff5b24 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:17.754037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:17.762321 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f32931a-5739-4d82-a234-f79c4e5ca755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:17.763230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32931a-5739-4d82-a234-f79c4e5ca755 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:17.763578 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32931a-5739-4d82-a234-f79c4e5ca755 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:17.763886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32931a-5739-4d82-a234-f79c4e5ca755 None admin] PATH_INFO: `/v3/projects/938e609f0dbd4b54b509fd8c09d15796` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:17.765130 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f32931a-5739-4d82-a234-f79c4e5ca755 None admin] RBAC: Authorizing `identity:get_project(project_id=938e609f0dbd4b54b509fd8c09d15796)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:17.767316 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f32931a-5739-4d82-a234-f79c4e5ca755 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:17.769090 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 558/1105] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:51:17 2022] GET /identity/v3/projects/938e609f0dbd4b54b509fd8c09d15796 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:17.775202 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d302bf-322e-4d37-a434-fd096f5f4d42 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:17.823070 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:17.872074 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d302bf-322e-4d37-a434-fd096f5f4d42 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:17.904800 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e2445-f208-4c6b-be23-35d8380d58b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:17.905380 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e2445-f208-4c6b-be23-35d8380d58b2 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:17.905664 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e2445-f208-4c6b-be23-35d8380d58b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:17.905947 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e2445-f208-4c6b-be23-35d8380d58b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:17.906797 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96e2445-f208-4c6b-be23-35d8380d58b2 None admin] RBAC: Adding query filter params (role.id=6b483758dbd44eb4a78df2ce4603d601, scope.project.id=938e609f0dbd4b54b509fd8c09d15796, user.id=4a284674692942e69800e67b57bbdaf2) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:51:17.907107 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96e2445-f208-4c6b-be23-35d8380d58b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:17.908632 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96e2445-f208-4c6b-be23-35d8380d58b2 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:17.914270 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 548/1106] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:51:17 2022] GET /identity/v3/role_assignments?user.id=4a284674692942e69800e67b57bbdaf2&scope.project.id=938e609f0dbd4b54b509fd8c09d15796&role.id=6b483758dbd44eb4a78df2ce4603d601 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:20.173819 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1108f-c2fc-4c5b-94c8-0645b42c0c73 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:20.174352 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1108f-c2fc-4c5b-94c8-0645b42c0c73 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:20.174640 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1108f-c2fc-4c5b-94c8-0645b42c0c73 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:20.176861 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 559/1107] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:51:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:51:20.182194 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92eddc-f1af-4eb3-9cd7-737335065476 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:20.182597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92eddc-f1af-4eb3-9cd7-737335065476 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:20.182750 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92eddc-f1af-4eb3-9cd7-737335065476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:20.205499 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-4e92eddc-f1af-4eb3-9cd7-737335065476 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:51:20.206692 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-4e92eddc-f1af-4eb3-9cd7-737335065476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:51:20.272080 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 549/1108] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:51:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:51:20.297629 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c1108f-c2fc-4c5b-94c8-0645b42c0c73 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:20.394893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c1108f-c2fc-4c5b-94c8-0645b42c0c73 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:20.427003 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b94fa9-e98c-4a1b-a4b3-95365a9ed066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:20.428056 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b94fa9-e98c-4a1b-a4b3-95365a9ed066 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:20.428553 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b94fa9-e98c-4a1b-a4b3-95365a9ed066 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:20.428862 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b94fa9-e98c-4a1b-a4b3-95365a9ed066 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:20.430028 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b94fa9-e98c-4a1b-a4b3-95365a9ed066 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:20.431476 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b94fa9-e98c-4a1b-a4b3-95365a9ed066 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:20.433052 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 560/1109] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:51:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:20.439281 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e92eddc-f1af-4eb3-9cd7-737335065476 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:20.486503 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:20.540783 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e92eddc-f1af-4eb3-9cd7-737335065476 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:20.575065 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b2c82c-0f9c-4e5a-b542-089df925febd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:20.575788 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2c82c-0f9c-4e5a-b542-089df925febd None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:20.576115 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2c82c-0f9c-4e5a-b542-089df925febd None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:20.576419 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2c82c-0f9c-4e5a-b542-089df925febd None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:20.577147 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b2c82c-0f9c-4e5a-b542-089df925febd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:20.579372 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b2c82c-0f9c-4e5a-b542-089df925febd None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:20.598262 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 550/1110] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Jun 7 21:51:20 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 21:51:22.876202 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512ea9a-2e87-49e6-841c-ec4cd51181c3 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:22.876713 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512ea9a-2e87-49e6-841c-ec4cd51181c3 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:22.876802 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512ea9a-2e87-49e6-841c-ec4cd51181c3 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:22.878116 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 561/1111] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:51:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:51:22.883239 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a23eb6-e8fb-4f30-9b1f-1d309204b4fc None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:22.883676 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a23eb6-e8fb-4f30-9b1f-1d309204b4fc None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:22.883826 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a23eb6-e8fb-4f30-9b1f-1d309204b4fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:22.900638 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-80a23eb6-e8fb-4f30-9b1f-1d309204b4fc None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:51:22.901803 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-80a23eb6-e8fb-4f30-9b1f-1d309204b4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:51:22.968295 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 551/1112] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:51:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:51:22.993322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5512ea9a-2e87-49e6-841c-ec4cd51181c3 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:23.054982 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:23.092060 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5512ea9a-2e87-49e6-841c-ec4cd51181c3 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:23.128881 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa093d-03c1-461b-a8f7-8c0c79327839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:23.130140 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa093d-03c1-461b-a8f7-8c0c79327839 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:23.130429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa093d-03c1-461b-a8f7-8c0c79327839 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:23.130858 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa093d-03c1-461b-a8f7-8c0c79327839 None admin] PATH_INFO: `/v3/roles/50e04e9d09e045b0bcf8e1782a6e2f23` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:23.132927 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fa093d-03c1-461b-a8f7-8c0c79327839 None admin] RBAC: Authorizing `identity:get_role(role_id=50e04e9d09e045b0bcf8e1782a6e2f23)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:23.134305 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fa093d-03c1-461b-a8f7-8c0c79327839 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:23.135701 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 562/1113] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:51:22 2022] GET /identity/v3/roles/50e04e9d09e045b0bcf8e1782a6e2f23 => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:23.142329 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a23eb6-e8fb-4f30-9b1f-1d309204b4fc None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:23.171868 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:23.239576 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a23eb6-e8fb-4f30-9b1f-1d309204b4fc None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:23.275646 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da76fe-41bf-47ca-b6a7-1dedafc4eab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:23.276515 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da76fe-41bf-47ca-b6a7-1dedafc4eab1 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:23.276798 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da76fe-41bf-47ca-b6a7-1dedafc4eab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:23.277074 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da76fe-41bf-47ca-b6a7-1dedafc4eab1 None admin] PATH_INFO: `/v3/users/0d3d0b35837547b1b9980281bc183f7a` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:23.290871 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7da76fe-41bf-47ca-b6a7-1dedafc4eab1 None admin] RBAC: Authorizing `identity:get_user(user_id=0d3d0b35837547b1b9980281bc183f7a)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:23.292446 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7da76fe-41bf-47ca-b6a7-1dedafc4eab1 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:23.294087 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 552/1114] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:51:23 2022] GET /identity/v3/users/0d3d0b35837547b1b9980281bc183f7a => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:23.300196 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa093d-03c1-461b-a8f7-8c0c79327839 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:23.397278 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa093d-03c1-461b-a8f7-8c0c79327839 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:23.425510 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:23.430705 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28e0c31-8da7-4d87-8fd6-98e8a32168a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:23.431363 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e0c31-8da7-4d87-8fd6-98e8a32168a8 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:23.431650 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e0c31-8da7-4d87-8fd6-98e8a32168a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:23.431926 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e0c31-8da7-4d87-8fd6-98e8a32168a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:23.498324 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28e0c31-8da7-4d87-8fd6-98e8a32168a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:23.499796 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28e0c31-8da7-4d87-8fd6-98e8a32168a8 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:23.596272 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 563/1115] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:51:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 21:51:23.602735 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da76fe-41bf-47ca-b6a7-1dedafc4eab1 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:23.689601 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:23.701350 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da76fe-41bf-47ca-b6a7-1dedafc4eab1 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:23.735344 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391e031-a999-4065-a394-b7011b022548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:23.736204 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391e031-a999-4065-a394-b7011b022548 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:23.736484 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391e031-a999-4065-a394-b7011b022548 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:23.736764 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391e031-a999-4065-a394-b7011b022548 None admin] PATH_INFO: `/v3/projects/938e609f0dbd4b54b509fd8c09d15796` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:23.737920 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c391e031-a999-4065-a394-b7011b022548 None admin] RBAC: Authorizing `identity:get_project(project_id=938e609f0dbd4b54b509fd8c09d15796)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:23.739347 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c391e031-a999-4065-a394-b7011b022548 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:23.741009 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 553/1116] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:51:23 2022] GET /identity/v3/projects/938e609f0dbd4b54b509fd8c09d15796 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:23.747036 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28e0c31-8da7-4d87-8fd6-98e8a32168a8 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:23.778442 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:23.844691 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28e0c31-8da7-4d87-8fd6-98e8a32168a8 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:23.878062 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87f0719-6ef0-4f4f-9863-dd23b71db563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:23.878706 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f0719-6ef0-4f4f-9863-dd23b71db563 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:23.879027 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f0719-6ef0-4f4f-9863-dd23b71db563 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:23.879483 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f0719-6ef0-4f4f-9863-dd23b71db563 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:23.880789 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87f0719-6ef0-4f4f-9863-dd23b71db563 None admin] RBAC: Adding query filter params (role.id=50e04e9d09e045b0bcf8e1782a6e2f23, scope.project.id=938e609f0dbd4b54b509fd8c09d15796, user.id=0d3d0b35837547b1b9980281bc183f7a) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:51:23.881262 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87f0719-6ef0-4f4f-9863-dd23b71db563 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:23.883519 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87f0719-6ef0-4f4f-9863-dd23b71db563 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:23.891815 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 564/1117] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:51:23 2022] GET /identity/v3/role_assignments?user.id=0d3d0b35837547b1b9980281bc183f7a&scope.project.id=938e609f0dbd4b54b509fd8c09d15796&role.id=50e04e9d09e045b0bcf8e1782a6e2f23 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:26.099222 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d5211-6071-4b6c-8fbc-714fbab1f980 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:26.100235 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d5211-6071-4b6c-8fbc-714fbab1f980 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:26.100235 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d5211-6071-4b6c-8fbc-714fbab1f980 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:26.101680 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 554/1118] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:51:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:51:26.106341 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24079f-1c74-4923-9cef-49fafe312c7e None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:26.106755 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24079f-1c74-4923-9cef-49fafe312c7e None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:26.106923 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24079f-1c74-4923-9cef-49fafe312c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:26.125790 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-bf24079f-1c74-4923-9cef-49fafe312c7e None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:51:26.127083 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-bf24079f-1c74-4923-9cef-49fafe312c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:51:26.194473 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 565/1119] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:51:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:51:26.219140 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5d5211-6071-4b6c-8fbc-714fbab1f980 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:26.319816 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5d5211-6071-4b6c-8fbc-714fbab1f980 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:26.353404 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab6cc7a-afe2-49e8-af6d-edc2bfd04d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:26.354191 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab6cc7a-afe2-49e8-af6d-edc2bfd04d0c None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:26.354477 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab6cc7a-afe2-49e8-af6d-edc2bfd04d0c None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:26.354761 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab6cc7a-afe2-49e8-af6d-edc2bfd04d0c None admin] PATH_INFO: `/v3/roles/50e04e9d09e045b0bcf8e1782a6e2f23` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:26.356315 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab6cc7a-afe2-49e8-af6d-edc2bfd04d0c None admin] RBAC: Authorizing `identity:get_role(role_id=50e04e9d09e045b0bcf8e1782a6e2f23)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:26.357675 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab6cc7a-afe2-49e8-af6d-edc2bfd04d0c None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:26.359021 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 555/1120] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:51:26 2022] GET /identity/v3/roles/50e04e9d09e045b0bcf8e1782a6e2f23 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:26.365138 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf24079f-1c74-4923-9cef-49fafe312c7e None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:26.391770 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:26.460398 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf24079f-1c74-4923-9cef-49fafe312c7e None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:26.492960 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f125ba00-ee2d-44c4-89e6-14e2160a3f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:26.493839 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f125ba00-ee2d-44c4-89e6-14e2160a3f0d None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:26.494113 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f125ba00-ee2d-44c4-89e6-14e2160a3f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:26.494381 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f125ba00-ee2d-44c4-89e6-14e2160a3f0d None admin] PATH_INFO: `/v3/users/0d3d0b35837547b1b9980281bc183f7a` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:26.495568 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f125ba00-ee2d-44c4-89e6-14e2160a3f0d None admin] RBAC: Authorizing `identity:get_user(user_id=0d3d0b35837547b1b9980281bc183f7a)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:26.497014 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f125ba00-ee2d-44c4-89e6-14e2160a3f0d None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:26.498598 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 566/1121] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:51:26 2022] GET /identity/v3/users/0d3d0b35837547b1b9980281bc183f7a => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:26.505201 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab6cc7a-afe2-49e8-af6d-edc2bfd04d0c None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:26.530395 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:26.603819 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab6cc7a-afe2-49e8-af6d-edc2bfd04d0c None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:26.636791 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac84a01-ef2f-4ff6-bbf3-6083ae36b0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:26.637634 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac84a01-ef2f-4ff6-bbf3-6083ae36b0b9 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:26.637917 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac84a01-ef2f-4ff6-bbf3-6083ae36b0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:26.638196 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac84a01-ef2f-4ff6-bbf3-6083ae36b0b9 None admin] PATH_INFO: `/v3/projects/938e609f0dbd4b54b509fd8c09d15796` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:26.639381 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac84a01-ef2f-4ff6-bbf3-6083ae36b0b9 None admin] RBAC: Authorizing `identity:get_project(project_id=938e609f0dbd4b54b509fd8c09d15796)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:26.641355 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac84a01-ef2f-4ff6-bbf3-6083ae36b0b9 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:26.643012 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 556/1122] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:51:26 2022] GET /identity/v3/projects/938e609f0dbd4b54b509fd8c09d15796 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:26.648920 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f125ba00-ee2d-44c4-89e6-14e2160a3f0d None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:26.739773 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:26.746353 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f125ba00-ee2d-44c4-89e6-14e2160a3f0d None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:26.781654 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dde160-83fb-4912-8f10-f8a2d378c4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:26.782532 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dde160-83fb-4912-8f10-f8a2d378c4a1 None admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:26.782814 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dde160-83fb-4912-8f10-f8a2d378c4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:26.783085 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dde160-83fb-4912-8f10-f8a2d378c4a1 None admin] PATH_INFO: `/v3/projects/938e609f0dbd4b54b509fd8c09d15796/users/0d3d0b35837547b1b9980281bc183f7a/roles/50e04e9d09e045b0bcf8e1782a6e2f23` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:26.785271 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dde160-83fb-4912-8f10-f8a2d378c4a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=938e609f0dbd4b54b509fd8c09d15796, user_id=0d3d0b35837547b1b9980281bc183f7a, role_id=50e04e9d09e045b0bcf8e1782a6e2f23)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:26.786679 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dde160-83fb-4912-8f10-f8a2d378c4a1 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:26.799610 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 567/1123] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jun 7 21:51:26 2022] PUT /identity/v3/projects/938e609f0dbd4b54b509fd8c09d15796/users/0d3d0b35837547b1b9980281bc183f7a/roles/50e04e9d09e045b0bcf8e1782a6e2f23 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 21:51:29.028825 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc1c89-35b3-48f5-8b38-6dfc1ace833c None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:29.029328 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc1c89-35b3-48f5-8b38-6dfc1ace833c None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:29.029328 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc1c89-35b3-48f5-8b38-6dfc1ace833c None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:29.030658 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 557/1124] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:51:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:51:29.036375 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6442273-d685-4161-bd26-097c3cdbe098 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:29.036785 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6442273-d685-4161-bd26-097c3cdbe098 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:29.036924 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6442273-d685-4161-bd26-097c3cdbe098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:29.054262 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-e6442273-d685-4161-bd26-097c3cdbe098 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:51:29.055459 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-e6442273-d685-4161-bd26-097c3cdbe098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:51:29.120563 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 568/1125] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:51:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:51:29.145153 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfc1c89-35b3-48f5-8b38-6dfc1ace833c None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:29.234486 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:29.244084 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfc1c89-35b3-48f5-8b38-6dfc1ace833c None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:29.276524 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ee4d49-a173-48ad-b58e-5115357ad809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:29.277369 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee4d49-a173-48ad-b58e-5115357ad809 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:29.277649 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee4d49-a173-48ad-b58e-5115357ad809 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:29.277922 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee4d49-a173-48ad-b58e-5115357ad809 None admin] PATH_INFO: `/v3/roles/50e04e9d09e045b0bcf8e1782a6e2f23` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:29.279427 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ee4d49-a173-48ad-b58e-5115357ad809 None admin] RBAC: Authorizing `identity:get_role(role_id=50e04e9d09e045b0bcf8e1782a6e2f23)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:29.280731 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ee4d49-a173-48ad-b58e-5115357ad809 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:29.283650 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 558/1126] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:51:29 2022] GET /identity/v3/roles/50e04e9d09e045b0bcf8e1782a6e2f23 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:29.291581 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6442273-d685-4161-bd26-097c3cdbe098 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:29.392990 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:29.409632 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6442273-d685-4161-bd26-097c3cdbe098 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:29.442048 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3442e1-40e5-41f1-9340-a267a0d94a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:29.442907 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3442e1-40e5-41f1-9340-a267a0d94a77 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:29.443185 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3442e1-40e5-41f1-9340-a267a0d94a77 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:29.443483 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3442e1-40e5-41f1-9340-a267a0d94a77 None admin] PATH_INFO: `/v3/users/0d3d0b35837547b1b9980281bc183f7a` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:29.444648 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3442e1-40e5-41f1-9340-a267a0d94a77 None admin] RBAC: Authorizing `identity:get_user(user_id=0d3d0b35837547b1b9980281bc183f7a)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:29.446074 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3442e1-40e5-41f1-9340-a267a0d94a77 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:29.447707 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 569/1127] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:51:29 2022] GET /identity/v3/users/0d3d0b35837547b1b9980281bc183f7a => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:29.454191 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ee4d49-a173-48ad-b58e-5115357ad809 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:29.550890 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ee4d49-a173-48ad-b58e-5115357ad809 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:29.585525 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c330ec9-a832-48d4-a35b-524515a3da7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:29.586159 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c330ec9-a832-48d4-a35b-524515a3da7d None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:29.586441 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c330ec9-a832-48d4-a35b-524515a3da7d None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:29.586716 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c330ec9-a832-48d4-a35b-524515a3da7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:29.613208 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:29.653007 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c330ec9-a832-48d4-a35b-524515a3da7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:29.654433 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c330ec9-a832-48d4-a35b-524515a3da7d None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:29.752618 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 559/1128] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:51:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 21:51:29.758594 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3442e1-40e5-41f1-9340-a267a0d94a77 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:29.854058 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3442e1-40e5-41f1-9340-a267a0d94a77 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:29.887523 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6579ea4b-a9fb-444a-8530-940b4a29aa73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:29.888366 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579ea4b-a9fb-444a-8530-940b4a29aa73 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:29.888646 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579ea4b-a9fb-444a-8530-940b4a29aa73 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:29.888918 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579ea4b-a9fb-444a-8530-940b4a29aa73 None admin] PATH_INFO: `/v3/projects/938e609f0dbd4b54b509fd8c09d15796` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:29.890076 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6579ea4b-a9fb-444a-8530-940b4a29aa73 None admin] RBAC: Authorizing `identity:get_project(project_id=938e609f0dbd4b54b509fd8c09d15796)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:29.891835 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6579ea4b-a9fb-444a-8530-940b4a29aa73 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:29.893487 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 570/1129] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:51:29 2022] GET /identity/v3/projects/938e609f0dbd4b54b509fd8c09d15796 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:29.899501 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c330ec9-a832-48d4-a35b-524515a3da7d None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:29.956784 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:29.998552 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c330ec9-a832-48d4-a35b-524515a3da7d None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:30.031632 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ea1b1-0b97-4cce-a714-edc7d0111a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:30.032356 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ea1b1-0b97-4cce-a714-edc7d0111a7f None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:30.032640 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ea1b1-0b97-4cce-a714-edc7d0111a7f None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:30.032915 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ea1b1-0b97-4cce-a714-edc7d0111a7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:30.033732 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208ea1b1-0b97-4cce-a714-edc7d0111a7f None admin] RBAC: Adding query filter params (role.id=50e04e9d09e045b0bcf8e1782a6e2f23, scope.project.id=938e609f0dbd4b54b509fd8c09d15796, user.id=0d3d0b35837547b1b9980281bc183f7a) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:51:30.034042 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208ea1b1-0b97-4cce-a714-edc7d0111a7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:30.035542 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208ea1b1-0b97-4cce-a714-edc7d0111a7f None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:30.041120 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 560/1130] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:51:29 2022] GET /identity/v3/role_assignments?user.id=0d3d0b35837547b1b9980281bc183f7a&scope.project.id=938e609f0dbd4b54b509fd8c09d15796&role.id=50e04e9d09e045b0bcf8e1782a6e2f23 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:32.311882 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d71d8e8-eb7d-4a84-8eff-14a121d16a0d None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:32.313077 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d71d8e8-eb7d-4a84-8eff-14a121d16a0d None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:32.313077 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d71d8e8-eb7d-4a84-8eff-14a121d16a0d None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:32.314417 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 571/1131] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:51:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:51:32.319656 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e8bd36-8f45-4c74-90f7-9edf040fd42d None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:32.320073 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e8bd36-8f45-4c74-90f7-9edf040fd42d None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:32.320214 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e8bd36-8f45-4c74-90f7-9edf040fd42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:32.336631 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-03e8bd36-8f45-4c74-90f7-9edf040fd42d None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:51:32.337812 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-03e8bd36-8f45-4c74-90f7-9edf040fd42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:51:32.402628 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 561/1132] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:51:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:51:32.426966 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d71d8e8-eb7d-4a84-8eff-14a121d16a0d None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:32.448510 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:32.526038 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d71d8e8-eb7d-4a84-8eff-14a121d16a0d None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:32.559380 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9111224-388e-4589-bc77-6e42d54c0ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:32.560154 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9111224-388e-4589-bc77-6e42d54c0ec9 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:32.560437 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9111224-388e-4589-bc77-6e42d54c0ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:32.560717 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9111224-388e-4589-bc77-6e42d54c0ec9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:32.561873 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9111224-388e-4589-bc77-6e42d54c0ec9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:32.563482 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9111224-388e-4589-bc77-6e42d54c0ec9 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:32.565011 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 572/1133] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:51:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:32.570808 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e8bd36-8f45-4c74-90f7-9edf040fd42d None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:32.624104 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:32.666872 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e8bd36-8f45-4c74-90f7-9edf040fd42d None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:32.699902 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de8c49f-f42a-4873-a6aa-24b3699869c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:32.700550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de8c49f-f42a-4873-a6aa-24b3699869c3 None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:32.700833 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de8c49f-f42a-4873-a6aa-24b3699869c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:32.701109 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de8c49f-f42a-4873-a6aa-24b3699869c3 None admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:32.701826 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de8c49f-f42a-4873-a6aa-24b3699869c3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:32.703511 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de8c49f-f42a-4873-a6aa-24b3699869c3 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:32.728314 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 562/1134] 10.222.0.13 () {60 vars in 1141 bytes} [Tue Jun 7 21:51:32 2022] POST /identity/v3/projects => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 21:51:35.026901 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ece0c4-48d3-414f-81b9-68ff6d47ac2e None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:35.027440 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ece0c4-48d3-414f-81b9-68ff6d47ac2e None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:35.027440 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ece0c4-48d3-414f-81b9-68ff6d47ac2e None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:35.028743 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 573/1135] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:51:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:51:35.034535 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb236f6-ea4c-462d-a654-4d600de8c27d None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:35.034966 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb236f6-ea4c-462d-a654-4d600de8c27d None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:35.035092 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb236f6-ea4c-462d-a654-4d600de8c27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:35.052057 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-ecb236f6-ea4c-462d-a654-4d600de8c27d None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:51:35.053219 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-ecb236f6-ea4c-462d-a654-4d600de8c27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:51:35.118341 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 563/1136] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:51:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:51:35.143240 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ece0c4-48d3-414f-81b9-68ff6d47ac2e None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:35.229123 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:35.241866 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ece0c4-48d3-414f-81b9-68ff6d47ac2e None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:35.274510 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78bee0f-0c9d-4ae4-ab75-59dacf9142fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:35.275288 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78bee0f-0c9d-4ae4-ab75-59dacf9142fa None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:35.275593 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78bee0f-0c9d-4ae4-ab75-59dacf9142fa None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:35.275865 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78bee0f-0c9d-4ae4-ab75-59dacf9142fa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:35.277073 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78bee0f-0c9d-4ae4-ab75-59dacf9142fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:35.278447 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78bee0f-0c9d-4ae4-ab75-59dacf9142fa None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:35.280065 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 574/1137] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:51:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:35.286543 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb236f6-ea4c-462d-a654-4d600de8c27d None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:35.319008 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:35.384210 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb236f6-ea4c-462d-a654-4d600de8c27d None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:35.417563 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c2eb20-5f8d-4052-8126-df8506f95285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:35.418222 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2eb20-5f8d-4052-8126-df8506f95285 None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:35.418503 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2eb20-5f8d-4052-8126-df8506f95285 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:35.418776 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2eb20-5f8d-4052-8126-df8506f95285 None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:35.419505 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c2eb20-5f8d-4052-8126-df8506f95285 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:35.420937 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c2eb20-5f8d-4052-8126-df8506f95285 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:35.441014 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 564/1138] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Jun 7 21:51:35 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 21:51:37.693115 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d71a62b-2f58-4e92-9f37-34d29f4490e1 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:37.693664 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d71a62b-2f58-4e92-9f37-34d29f4490e1 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:37.693664 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d71a62b-2f58-4e92-9f37-34d29f4490e1 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:37.695016 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 575/1139] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:51:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:51:37.700178 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda94bc-36ba-49a6-9ab3-5c6b035f0c22 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:37.700634 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda94bc-36ba-49a6-9ab3-5c6b035f0c22 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:37.700750 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda94bc-36ba-49a6-9ab3-5c6b035f0c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:37.717965 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-5bda94bc-36ba-49a6-9ab3-5c6b035f0c22 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:51:37.719907 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-5bda94bc-36ba-49a6-9ab3-5c6b035f0c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:51:37.792062 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 565/1140] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:51:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 21:51:37.816165 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d71a62b-2f58-4e92-9f37-34d29f4490e1 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:37.920362 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d71a62b-2f58-4e92-9f37-34d29f4490e1 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:37.954471 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08bf36a-a264-4541-a6c0-082ce31511ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:37.955182 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08bf36a-a264-4541-a6c0-082ce31511ec None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:37.955498 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08bf36a-a264-4541-a6c0-082ce31511ec None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:37.955777 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08bf36a-a264-4541-a6c0-082ce31511ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:37.965563 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08bf36a-a264-4541-a6c0-082ce31511ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:51:37.973500 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-f08bf36a-a264-4541-a6c0-082ce31511ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:51:37.973500 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:51:37.973500 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:51:37.973500 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:51:37.973500 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:51:37.973500 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:51:37.973500 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:51:37.973500 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:51:37.973500 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:51:37.973500 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:51:37.973500 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:51:37.973500 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:51:37.973500 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:51:37.973500 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:51:37.973500 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:51:37.973500 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:51:37.974602 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:51:37.974602 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:51:37.974602 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:51:37.974602 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:51:37.974602 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:51:37.974602 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:51:37.974602 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:51:37.974602 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:51:37.974602 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:51:37.974602 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:51:37.974602 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:51:37.974602 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:51:37.974602 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:51:37.974602 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:51:37.974602 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:51:37.974602 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:51:37.975659 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:51:37.975659 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:51:37.975659 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:51:37.975659 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:51:37.975659 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:51:37.975659 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:51:37.975659 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:51:37.975659 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:51:37.975659 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:51:37.975659 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:51:37.975659 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:51:37.975659 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:51:37.975659 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 576/1141] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jun 7 21:51:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:51:37.980870 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bda94bc-36ba-49a6-9ab3-5c6b035f0c22 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:38.004879 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:38.078455 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bda94bc-36ba-49a6-9ab3-5c6b035f0c22 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:38.111431 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49637519-d2ad-4452-90d2-50834dc5bac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:38.112085 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49637519-d2ad-4452-90d2-50834dc5bac4 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:38.112368 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49637519-d2ad-4452-90d2-50834dc5bac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:38.112644 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49637519-d2ad-4452-90d2-50834dc5bac4 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:38.113384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49637519-d2ad-4452-90d2-50834dc5bac4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:51:38.113691 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49637519-d2ad-4452-90d2-50834dc5bac4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:38.115044 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49637519-d2ad-4452-90d2-50834dc5bac4 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:38.122856 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 566/1142] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:51:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:38.128594 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08bf36a-a264-4541-a6c0-082ce31511ec None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:38.147004 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:38.226710 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08bf36a-a264-4541-a6c0-082ce31511ec None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:38.257978 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4635014a-494f-4ee3-b812-8f0267ac8ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:38.258823 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635014a-494f-4ee3-b812-8f0267ac8ba9 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:38.259102 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635014a-494f-4ee3-b812-8f0267ac8ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:38.259398 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635014a-494f-4ee3-b812-8f0267ac8ba9 None admin] PATH_INFO: `/v3/users/0ce9f8d9dbc042468bef43a17c323692` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:38.272530 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4635014a-494f-4ee3-b812-8f0267ac8ba9 None admin] RBAC: Authorizing `identity:get_user(user_id=0ce9f8d9dbc042468bef43a17c323692)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:38.273959 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4635014a-494f-4ee3-b812-8f0267ac8ba9 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:38.275580 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 577/1143] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:51:38 2022] GET /identity/v3/users/0ce9f8d9dbc042468bef43a17c323692 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:38.281250 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49637519-d2ad-4452-90d2-50834dc5bac4 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:38.364146 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:38.379890 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49637519-d2ad-4452-90d2-50834dc5bac4 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:38.411626 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14344d7e-3e39-4373-a20e-88c354f12fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:38.412258 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14344d7e-3e39-4373-a20e-88c354f12fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:38.412537 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14344d7e-3e39-4373-a20e-88c354f12fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:38.412808 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14344d7e-3e39-4373-a20e-88c354f12fa0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:38.476883 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14344d7e-3e39-4373-a20e-88c354f12fa0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:38.478243 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14344d7e-3e39-4373-a20e-88c354f12fa0 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:38.555675 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:38.571978 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 567/1144] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:51:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 21:51:38.578267 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4635014a-494f-4ee3-b812-8f0267ac8ba9 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:38.649480 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:38.673522 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4635014a-494f-4ee3-b812-8f0267ac8ba9 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:38.705653 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da88e6-3d69-4884-ac75-6855f69b80f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:38.706494 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da88e6-3d69-4884-ac75-6855f69b80f4 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:38.706773 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da88e6-3d69-4884-ac75-6855f69b80f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:38.707044 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da88e6-3d69-4884-ac75-6855f69b80f4 None admin] PATH_INFO: `/v3/projects/d797de216dcb4190aae4c365e8be7c59` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:38.708222 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2da88e6-3d69-4884-ac75-6855f69b80f4 None admin] RBAC: Authorizing `identity:get_project(project_id=d797de216dcb4190aae4c365e8be7c59)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:38.709641 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2da88e6-3d69-4884-ac75-6855f69b80f4 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:38.711273 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 578/1145] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:51:38 2022] GET /identity/v3/projects/d797de216dcb4190aae4c365e8be7c59 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:38.717265 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14344d7e-3e39-4373-a20e-88c354f12fa0 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:38.815232 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14344d7e-3e39-4373-a20e-88c354f12fa0 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:38.848071 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ccf574-d349-4134-8ec3-71d3a308bec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:38.848717 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ccf574-d349-4134-8ec3-71d3a308bec2 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:38.848999 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ccf574-d349-4134-8ec3-71d3a308bec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:38.849274 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ccf574-d349-4134-8ec3-71d3a308bec2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:38.850094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ccf574-d349-4134-8ec3-71d3a308bec2 None admin] RBAC: Adding query filter params (role.id=6b483758dbd44eb4a78df2ce4603d601, scope.project.id=d797de216dcb4190aae4c365e8be7c59, user.id=0ce9f8d9dbc042468bef43a17c323692) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:51:38.850402 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ccf574-d349-4134-8ec3-71d3a308bec2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:38.851852 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ccf574-d349-4134-8ec3-71d3a308bec2 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:38.857084 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 568/1146] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:51:38 2022] GET /identity/v3/role_assignments?user.id=0ce9f8d9dbc042468bef43a17c323692&scope.project.id=d797de216dcb4190aae4c365e8be7c59&role.id=6b483758dbd44eb4a78df2ce4603d601 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:41.063565 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f737a943-60e2-4314-8a37-8b9c61179b4d None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:41.064147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f737a943-60e2-4314-8a37-8b9c61179b4d None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:41.064147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f737a943-60e2-4314-8a37-8b9c61179b4d None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:41.065456 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 579/1147] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:51:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:51:41.070570 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5509e-d23d-4307-96a3-793d92b80bc6 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:41.070957 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5509e-d23d-4307-96a3-793d92b80bc6 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:41.071126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5509e-d23d-4307-96a3-793d92b80bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:41.087980 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-40a5509e-d23d-4307-96a3-793d92b80bc6 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:51:41.089160 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-40a5509e-d23d-4307-96a3-793d92b80bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:51:41.132312 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:41.155075 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 569/1148] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:51:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:51:41.178422 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f737a943-60e2-4314-8a37-8b9c61179b4d None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:41.278159 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f737a943-60e2-4314-8a37-8b9c61179b4d None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:41.311182 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9643adca-4d02-4c83-9bb1-8ea7ccc514c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:41.312721 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643adca-4d02-4c83-9bb1-8ea7ccc514c1 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:41.313149 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643adca-4d02-4c83-9bb1-8ea7ccc514c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:41.313428 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643adca-4d02-4c83-9bb1-8ea7ccc514c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:41.323153 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9643adca-4d02-4c83-9bb1-8ea7ccc514c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:51:41.330944 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-9643adca-4d02-4c83-9bb1-8ea7ccc514c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:51:41.330944 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:51:41.330944 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:51:41.330944 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:51:41.330944 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:51:41.330944 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:51:41.330944 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:51:41.330944 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:51:41.330944 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:51:41.330944 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:51:41.330944 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:51:41.330944 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:51:41.330944 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:51:41.330944 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:51:41.330944 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:51:41.330944 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:51:41.332154 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:51:41.332154 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:51:41.332154 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:51:41.332154 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:51:41.332154 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:51:41.332154 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:51:41.332154 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:51:41.332154 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:51:41.332154 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:51:41.332154 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:51:41.332154 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:51:41.332154 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:51:41.332154 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:51:41.332154 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:51:41.332154 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:51:41.332154 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:51:41.333085 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:51:41.333085 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:51:41.333085 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:51:41.333085 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:51:41.333085 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:51:41.333085 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:51:41.333085 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:51:41.333085 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:51:41.333085 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:51:41.333085 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:51:41.333085 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:51:41.333085 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:51:41.333085 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 580/1149] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jun 7 21:51:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:51:41.338115 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a5509e-d23d-4307-96a3-793d92b80bc6 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:41.433339 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a5509e-d23d-4307-96a3-793d92b80bc6 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:41.470115 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8035af-1a88-445c-83a5-e5ebd55c7069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:41.470785 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8035af-1a88-445c-83a5-e5ebd55c7069 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:41.471072 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8035af-1a88-445c-83a5-e5ebd55c7069 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:41.471375 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8035af-1a88-445c-83a5-e5ebd55c7069 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:41.472384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8035af-1a88-445c-83a5-e5ebd55c7069 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:51:41.472693 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8035af-1a88-445c-83a5-e5ebd55c7069 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:41.474049 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8035af-1a88-445c-83a5-e5ebd55c7069 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:41.481945 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 570/1150] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:51:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:41.488403 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9643adca-4d02-4c83-9bb1-8ea7ccc514c1 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:41.491252 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:41.586955 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9643adca-4d02-4c83-9bb1-8ea7ccc514c1 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:41.619881 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae135b-d8e3-420c-86f4-163c10a3bfc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:41.620740 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae135b-d8e3-420c-86f4-163c10a3bfc7 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:41.621024 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae135b-d8e3-420c-86f4-163c10a3bfc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:41.621299 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae135b-d8e3-420c-86f4-163c10a3bfc7 None admin] PATH_INFO: `/v3/users/0ce9f8d9dbc042468bef43a17c323692` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:41.622478 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beae135b-d8e3-420c-86f4-163c10a3bfc7 None admin] RBAC: Authorizing `identity:get_user(user_id=0ce9f8d9dbc042468bef43a17c323692)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:41.623952 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beae135b-d8e3-420c-86f4-163c10a3bfc7 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:41.625868 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 581/1151] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:51:41 2022] GET /identity/v3/users/0ce9f8d9dbc042468bef43a17c323692 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:41.631536 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8035af-1a88-445c-83a5-e5ebd55c7069 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:41.673091 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:41.734536 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8035af-1a88-445c-83a5-e5ebd55c7069 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:41.767205 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d74e19d-61ca-4e42-8a36-53048b2ea70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:41.768096 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74e19d-61ca-4e42-8a36-53048b2ea70b None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:41.768381 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74e19d-61ca-4e42-8a36-53048b2ea70b None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:41.768658 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74e19d-61ca-4e42-8a36-53048b2ea70b None admin] PATH_INFO: `/v3/projects/d797de216dcb4190aae4c365e8be7c59` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:41.769846 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d74e19d-61ca-4e42-8a36-53048b2ea70b None admin] RBAC: Authorizing `identity:get_project(project_id=d797de216dcb4190aae4c365e8be7c59)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:41.771276 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d74e19d-61ca-4e42-8a36-53048b2ea70b None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:41.773001 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 571/1152] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:51:41 2022] GET /identity/v3/projects/d797de216dcb4190aae4c365e8be7c59 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:41.778964 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae135b-d8e3-420c-86f4-163c10a3bfc7 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:41.846186 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:41.877403 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae135b-d8e3-420c-86f4-163c10a3bfc7 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:41.909880 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017eb324-2510-4acd-8b01-900d8019c6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:41.910755 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017eb324-2510-4acd-8b01-900d8019c6a7 None admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:41.911036 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017eb324-2510-4acd-8b01-900d8019c6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:41.911317 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017eb324-2510-4acd-8b01-900d8019c6a7 None admin] PATH_INFO: `/v3/projects/d797de216dcb4190aae4c365e8be7c59/users/0ce9f8d9dbc042468bef43a17c323692/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:41.913262 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017eb324-2510-4acd-8b01-900d8019c6a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=d797de216dcb4190aae4c365e8be7c59, user_id=0ce9f8d9dbc042468bef43a17c323692, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:41.914679 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017eb324-2510-4acd-8b01-900d8019c6a7 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:41.927451 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 582/1153] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jun 7 21:51:41 2022] PUT /identity/v3/projects/d797de216dcb4190aae4c365e8be7c59/users/0ce9f8d9dbc042468bef43a17c323692/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 21:51:44.164362 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731bb714-de86-4db6-bf34-b6844b989468 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:44.165160 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731bb714-de86-4db6-bf34-b6844b989468 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:44.165315 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731bb714-de86-4db6-bf34-b6844b989468 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:44.166757 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 572/1154] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:51:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:51:44.172151 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648a0471-4d05-4313-b502-2dbb785e9af8 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:44.172582 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648a0471-4d05-4313-b502-2dbb785e9af8 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:44.172687 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648a0471-4d05-4313-b502-2dbb785e9af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:44.189062 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-648a0471-4d05-4313-b502-2dbb785e9af8 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:51:44.190230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-648a0471-4d05-4313-b502-2dbb785e9af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:51:44.255712 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 583/1155] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:51:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:51:44.279864 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731bb714-de86-4db6-bf34-b6844b989468 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:44.380068 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731bb714-de86-4db6-bf34-b6844b989468 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:44.388538 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:44.416841 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a274587-55af-4dc4-bb4f-7d5326f5f6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:44.417547 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a274587-55af-4dc4-bb4f-7d5326f5f6d3 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:44.417829 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a274587-55af-4dc4-bb4f-7d5326f5f6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:44.418103 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a274587-55af-4dc4-bb4f-7d5326f5f6d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:44.429099 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a274587-55af-4dc4-bb4f-7d5326f5f6d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:51:44.436528 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-7a274587-55af-4dc4-bb4f-7d5326f5f6d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:51:44.436528 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:51:44.436528 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:51:44.436528 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:51:44.436528 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:51:44.436528 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:51:44.436528 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:51:44.436528 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:51:44.436528 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:51:44.436528 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:51:44.436528 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:51:44.436528 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:51:44.436528 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:51:44.436528 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:51:44.436528 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:51:44.436528 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:51:44.437604 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:51:44.437604 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:51:44.437604 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:51:44.437604 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:51:44.437604 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:51:44.437604 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:51:44.437604 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:51:44.437604 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:51:44.437604 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:51:44.437604 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:51:44.437604 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:51:44.437604 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:51:44.437604 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:51:44.437604 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:51:44.437604 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:51:44.437604 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:51:44.438646 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:51:44.438646 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:51:44.438646 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:51:44.438646 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:51:44.438646 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:51:44.438646 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:51:44.438646 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:51:44.438646 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:51:44.438646 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:51:44.438646 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:51:44.438646 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:51:44.438646 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:51:44.438646 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 573/1156] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jun 7 21:51:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:51:44.444456 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648a0471-4d05-4313-b502-2dbb785e9af8 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:44.503083 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:44.542092 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648a0471-4d05-4313-b502-2dbb785e9af8 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:44.574653 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aa486d-f667-4bfe-8a55-f8717d3c1f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:44.575313 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa486d-f667-4bfe-8a55-f8717d3c1f21 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:44.575658 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa486d-f667-4bfe-8a55-f8717d3c1f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:44.575908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa486d-f667-4bfe-8a55-f8717d3c1f21 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:44.576655 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aa486d-f667-4bfe-8a55-f8717d3c1f21 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:51:44.576963 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aa486d-f667-4bfe-8a55-f8717d3c1f21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:44.578968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aa486d-f667-4bfe-8a55-f8717d3c1f21 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:44.586794 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 584/1157] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:51:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:44.592649 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a274587-55af-4dc4-bb4f-7d5326f5f6d3 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:44.690064 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a274587-55af-4dc4-bb4f-7d5326f5f6d3 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:44.724921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7c9d77-7f42-42bc-bd22-027a434caca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:44.725775 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c9d77-7f42-42bc-bd22-027a434caca5 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:44.726056 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c9d77-7f42-42bc-bd22-027a434caca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:44.726331 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c9d77-7f42-42bc-bd22-027a434caca5 None admin] PATH_INFO: `/v3/users/0ce9f8d9dbc042468bef43a17c323692` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:44.727538 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7c9d77-7f42-42bc-bd22-027a434caca5 None admin] RBAC: Authorizing `identity:get_user(user_id=0ce9f8d9dbc042468bef43a17c323692)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:44.728978 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7c9d77-7f42-42bc-bd22-027a434caca5 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:44.730592 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 574/1158] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:51:44 2022] GET /identity/v3/users/0ce9f8d9dbc042468bef43a17c323692 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:44.736618 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aa486d-f667-4bfe-8a55-f8717d3c1f21 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:44.836260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aa486d-f667-4bfe-8a55-f8717d3c1f21 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:44.860383 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:44.872895 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bdd97e-cd47-4e46-aea9-fd181fa16ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:44.873543 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bdd97e-cd47-4e46-aea9-fd181fa16ca7 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:44.873828 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bdd97e-cd47-4e46-aea9-fd181fa16ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:44.874100 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bdd97e-cd47-4e46-aea9-fd181fa16ca7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:44.942342 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bdd97e-cd47-4e46-aea9-fd181fa16ca7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:44.943739 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bdd97e-cd47-4e46-aea9-fd181fa16ca7 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:45.042602 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 585/1159] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:51:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 21:51:45.051523 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7c9d77-7f42-42bc-bd22-027a434caca5 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:45.085900 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:45.149435 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7c9d77-7f42-42bc-bd22-027a434caca5 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:45.182441 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cbd1eb-c380-42f3-b103-4025e9aaae06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:45.183288 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cbd1eb-c380-42f3-b103-4025e9aaae06 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:45.183597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cbd1eb-c380-42f3-b103-4025e9aaae06 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:45.183873 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cbd1eb-c380-42f3-b103-4025e9aaae06 None admin] PATH_INFO: `/v3/projects/d797de216dcb4190aae4c365e8be7c59` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:45.185039 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cbd1eb-c380-42f3-b103-4025e9aaae06 None admin] RBAC: Authorizing `identity:get_project(project_id=d797de216dcb4190aae4c365e8be7c59)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:45.186449 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cbd1eb-c380-42f3-b103-4025e9aaae06 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:45.188432 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 575/1160] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:51:45 2022] GET /identity/v3/projects/d797de216dcb4190aae4c365e8be7c59 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:45.194714 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bdd97e-cd47-4e46-aea9-fd181fa16ca7 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:45.220633 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:45.294099 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bdd97e-cd47-4e46-aea9-fd181fa16ca7 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:45.326573 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c33831-1afd-4857-abe6-475aecb62d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:45.327210 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c33831-1afd-4857-abe6-475aecb62d22 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:45.328212 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c33831-1afd-4857-abe6-475aecb62d22 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:45.328669 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c33831-1afd-4857-abe6-475aecb62d22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:45.329498 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c33831-1afd-4857-abe6-475aecb62d22 None admin] RBAC: Adding query filter params (role.id=6b483758dbd44eb4a78df2ce4603d601, scope.project.id=d797de216dcb4190aae4c365e8be7c59, user.id=0ce9f8d9dbc042468bef43a17c323692) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:51:45.329810 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c33831-1afd-4857-abe6-475aecb62d22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:45.331234 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c33831-1afd-4857-abe6-475aecb62d22 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:45.336889 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 586/1161] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:51:45 2022] GET /identity/v3/role_assignments?user.id=0ce9f8d9dbc042468bef43a17c323692&scope.project.id=d797de216dcb4190aae4c365e8be7c59&role.id=6b483758dbd44eb4a78df2ce4603d601 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:47.632640 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1458606-3413-44f8-89d1-92f1a0fee841 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:47.633199 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1458606-3413-44f8-89d1-92f1a0fee841 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:47.633199 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1458606-3413-44f8-89d1-92f1a0fee841 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:47.634564 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 576/1162] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:51:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:51:47.640559 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05773563-2f12-4ec2-8498-f4d5d3010951 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:47.640948 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05773563-2f12-4ec2-8498-f4d5d3010951 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:47.641207 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05773563-2f12-4ec2-8498-f4d5d3010951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:47.658747 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-05773563-2f12-4ec2-8498-f4d5d3010951 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:51:47.659966 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-05773563-2f12-4ec2-8498-f4d5d3010951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:51:47.745176 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 587/1163] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:51:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:51:47.773387 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1458606-3413-44f8-89d1-92f1a0fee841 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:47.873635 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1458606-3413-44f8-89d1-92f1a0fee841 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:47.876108 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:47.907683 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f1bbce-0de2-463d-8c5e-9a0649a6b5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:47.908261 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1bbce-0de2-463d-8c5e-9a0649a6b5fc None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:47.908541 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1bbce-0de2-463d-8c5e-9a0649a6b5fc None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:47.908814 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1bbce-0de2-463d-8c5e-9a0649a6b5fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:47.973034 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f1bbce-0de2-463d-8c5e-9a0649a6b5fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:47.974458 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f1bbce-0de2-463d-8c5e-9a0649a6b5fc None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:48.072791 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 577/1164] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:51:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 21:51:48.080257 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05773563-2f12-4ec2-8498-f4d5d3010951 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:48.104506 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:48.184914 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05773563-2f12-4ec2-8498-f4d5d3010951 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:48.217311 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293d0641-cd23-45fe-bed1-722e146968ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:48.218783 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d0641-cd23-45fe-bed1-722e146968ca None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:48.219069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d0641-cd23-45fe-bed1-722e146968ca None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:48.219368 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d0641-cd23-45fe-bed1-722e146968ca None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:48.226158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293d0641-cd23-45fe-bed1-722e146968ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:48.227642 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293d0641-cd23-45fe-bed1-722e146968ca None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:48.241436 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-293d0641-cd23-45fe-bed1-722e146968ca None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 07 21:51:48.241436 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:51:48.241436 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 07 21:51:48.241436 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 07 21:51:48.241436 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 21:51:48.241436 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 21:51:48.241436 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 21:51:48.241436 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 21:51:48.241436 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 07 21:51:48.241436 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:51:48.241436 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 21:51:48.241436 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:51:48.241436 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:51:48.241436 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:51:48.241436 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:51:48.241436 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:51:48.241436 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:51:48.242650 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:51:48.242650 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:51:48.242650 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:51:48.242650 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:51:48.242650 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:51:48.242650 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:51:48.242650 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:51:48.242650 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:51:48.242650 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:51:48.242650 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:51:48.242650 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:51:48.242650 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:51:48.242650 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:51:48.242650 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:51:48.242650 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:51:48.242650 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:51:48.243984 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:51:48.243984 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:51:48.243984 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:51:48.243984 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:51:48.243984 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:51:48.243984 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:51:48.243984 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:51:48.243984 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:51:48.243984 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:51:48.243984 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:51:48.243984 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 07 21:51:48.243984 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:51:48.243984 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 07 21:51:48.243984 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:51:48.243984 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 588/1165] 10.222.0.13 () {56 vars in 1117 bytes} [Tue Jun 7 21:51:48 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:51:48.248699 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f1bbce-0de2-463d-8c5e-9a0649a6b5fc None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:48.252623 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:48.347839 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f1bbce-0de2-463d-8c5e-9a0649a6b5fc None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:48.380953 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4327a4-0d92-4d25-8ddd-19d5b99d6769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:48.381612 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4327a4-0d92-4d25-8ddd-19d5b99d6769 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:48.381896 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4327a4-0d92-4d25-8ddd-19d5b99d6769 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:48.382174 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4327a4-0d92-4d25-8ddd-19d5b99d6769 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:48.382890 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4327a4-0d92-4d25-8ddd-19d5b99d6769 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:51:48.383197 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4327a4-0d92-4d25-8ddd-19d5b99d6769 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:48.384604 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4327a4-0d92-4d25-8ddd-19d5b99d6769 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:48.393057 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 578/1166] 10.222.0.13 () {56 vars in 1115 bytes} [Tue Jun 7 21:51:48 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:50.678112 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a4269-b2c6-452b-94ec-0cf6156d16f1 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:50.678646 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a4269-b2c6-452b-94ec-0cf6156d16f1 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:50.678737 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a4269-b2c6-452b-94ec-0cf6156d16f1 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:50.680063 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 589/1167] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:51:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:51:50.685239 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896d7d9-c305-4ffb-ad42-81179ce78458 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:50.685640 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896d7d9-c305-4ffb-ad42-81179ce78458 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:50.685807 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896d7d9-c305-4ffb-ad42-81179ce78458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:50.703561 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-6896d7d9-c305-4ffb-ad42-81179ce78458 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:51:50.704759 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-6896d7d9-c305-4ffb-ad42-81179ce78458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:51:50.764337 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:50.770527 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 579/1168] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:51:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 21:51:50.793369 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0a4269-b2c6-452b-94ec-0cf6156d16f1 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:50.868980 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:50.892628 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0a4269-b2c6-452b-94ec-0cf6156d16f1 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:50.924802 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534967e7-f203-4b50-8818-62b1f0e27f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:50.925445 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534967e7-f203-4b50-8818-62b1f0e27f67 None admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:50.925727 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534967e7-f203-4b50-8818-62b1f0e27f67 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:50.925999 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534967e7-f203-4b50-8818-62b1f0e27f67 None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:50.926726 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534967e7-f203-4b50-8818-62b1f0e27f67 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:50.928107 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534967e7-f203-4b50-8818-62b1f0e27f67 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:50.942507 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 590/1169] 10.222.0.13 () {60 vars in 1138 bytes} [Tue Jun 7 21:51:50 2022] POST /identity/v3/domains => generated 267 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 21:51:53.177966 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d6420b-1e41-4578-92c7-17adf8bae22e None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:53.178501 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d6420b-1e41-4578-92c7-17adf8bae22e None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:53.178501 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d6420b-1e41-4578-92c7-17adf8bae22e None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:53.179918 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 580/1170] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:51:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:51:53.185049 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c943bbda-dde5-4882-9414-71b6897036e3 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:53.185419 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c943bbda-dde5-4882-9414-71b6897036e3 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:53.185604 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c943bbda-dde5-4882-9414-71b6897036e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:53.202470 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-c943bbda-dde5-4882-9414-71b6897036e3 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:51:53.203652 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c943bbda-dde5-4882-9414-71b6897036e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:51:53.267113 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 591/1171] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:51:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:51:53.290288 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d6420b-1e41-4578-92c7-17adf8bae22e None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:53.392181 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d6420b-1e41-4578-92c7-17adf8bae22e None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:53.424875 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9448e783-8eeb-4358-8140-6ce528caf337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:53.425720 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9448e783-8eeb-4358-8140-6ce528caf337 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:53.426000 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9448e783-8eeb-4358-8140-6ce528caf337 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:53.426276 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9448e783-8eeb-4358-8140-6ce528caf337 None admin] PATH_INFO: `/v3/domains/0c70061feef24fc8b2e9585cdb491cec` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:53.437033 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9448e783-8eeb-4358-8140-6ce528caf337 None admin] RBAC: Authorizing `identity:get_domain(domain_id=0c70061feef24fc8b2e9585cdb491cec)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:53.438428 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9448e783-8eeb-4358-8140-6ce528caf337 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:53.440072 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 581/1172] 10.222.0.13 () {56 vars in 1183 bytes} [Tue Jun 7 21:51:53 2022] GET /identity/v3/domains/0c70061feef24fc8b2e9585cdb491cec => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:53.446022 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c943bbda-dde5-4882-9414-71b6897036e3 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:53.508713 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:53.541780 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c943bbda-dde5-4882-9414-71b6897036e3 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:53.574640 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b8b1b-f941-47af-9bb7-337fecc1d08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:53.575290 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b8b1b-f941-47af-9bb7-337fecc1d08f None admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:53.575603 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b8b1b-f941-47af-9bb7-337fecc1d08f None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:53.575875 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b8b1b-f941-47af-9bb7-337fecc1d08f None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:53.576585 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353b8b1b-f941-47af-9bb7-337fecc1d08f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:53.578045 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353b8b1b-f941-47af-9bb7-337fecc1d08f None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:53.603200 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 592/1173] 10.222.0.13 () {60 vars in 1141 bytes} [Tue Jun 7 21:51:53 2022] POST /identity/v3/projects => generated 361 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 21:51:55.879817 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8314e561-31e3-4fd4-901b-e679622aa6a1 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:55.880328 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8314e561-31e3-4fd4-901b-e679622aa6a1 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:55.880445 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8314e561-31e3-4fd4-901b-e679622aa6a1 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:55.881740 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 582/1174] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:51:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:51:55.887177 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d0d4c-8488-4abd-bd1d-215990f9c294 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:55.887574 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d0d4c-8488-4abd-bd1d-215990f9c294 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:55.887771 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d0d4c-8488-4abd-bd1d-215990f9c294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:55.904522 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-2f5d0d4c-8488-4abd-bd1d-215990f9c294 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:51:55.906222 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-2f5d0d4c-8488-4abd-bd1d-215990f9c294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:51:55.970648 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 593/1175] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:51:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:51:55.995939 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8314e561-31e3-4fd4-901b-e679622aa6a1 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:56.054360 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:56.095966 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8314e561-31e3-4fd4-901b-e679622aa6a1 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:56.128750 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb9e52a-f71b-43cd-9578-68048e0973f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:56.130291 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9e52a-f71b-43cd-9578-68048e0973f5 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:56.130291 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9e52a-f71b-43cd-9578-68048e0973f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:56.130291 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9e52a-f71b-43cd-9578-68048e0973f5 None admin] PATH_INFO: `/v3/domains/0c70061feef24fc8b2e9585cdb491cec` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:56.131319 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb9e52a-f71b-43cd-9578-68048e0973f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=0c70061feef24fc8b2e9585cdb491cec)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:56.132775 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb9e52a-f71b-43cd-9578-68048e0973f5 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:56.134373 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 583/1176] 10.222.0.13 () {56 vars in 1183 bytes} [Tue Jun 7 21:51:55 2022] GET /identity/v3/domains/0c70061feef24fc8b2e9585cdb491cec => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:56.140864 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d0d4c-8488-4abd-bd1d-215990f9c294 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:56.187928 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:56.237619 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d0d4c-8488-4abd-bd1d-215990f9c294 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:56.270206 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c7ba9f-2aeb-41cc-ae3a-f357718f190b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:56.270863 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7ba9f-2aeb-41cc-ae3a-f357718f190b None admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:56.271145 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7ba9f-2aeb-41cc-ae3a-f357718f190b None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:56.271445 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7ba9f-2aeb-41cc-ae3a-f357718f190b None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:56.272143 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c7ba9f-2aeb-41cc-ae3a-f357718f190b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:56.273575 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c7ba9f-2aeb-41cc-ae3a-f357718f190b None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:56.294271 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 594/1177] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Jun 7 21:51:56 2022] POST /identity/v3/users => generated 311 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 21:51:58.575513 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd57cba-81e4-466d-921d-091a5f4162dc None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:58.576089 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd57cba-81e4-466d-921d-091a5f4162dc None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:58.576089 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd57cba-81e4-466d-921d-091a5f4162dc None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:58.577986 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 584/1178] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:51:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:51:58.583266 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5657810-f6d4-47c4-a852-3a41873eec3c None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:58.583703 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5657810-f6d4-47c4-a852-3a41873eec3c None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:58.583859 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5657810-f6d4-47c4-a852-3a41873eec3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:58.601281 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-b5657810-f6d4-47c4-a852-3a41873eec3c None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:51:58.602486 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-b5657810-f6d4-47c4-a852-3a41873eec3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:51:58.666239 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 595/1179] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:51:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:51:58.689467 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd57cba-81e4-466d-921d-091a5f4162dc None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:58.789817 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd57cba-81e4-466d-921d-091a5f4162dc None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:58.814763 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:58.823448 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f0c7c-e79f-400b-aedc-bdd56a517d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:58.824231 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f0c7c-e79f-400b-aedc-bdd56a517d67 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:58.824514 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f0c7c-e79f-400b-aedc-bdd56a517d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:58.824787 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f0c7c-e79f-400b-aedc-bdd56a517d67 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:58.835830 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7f0c7c-e79f-400b-aedc-bdd56a517d67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:51:58.843555 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-0e7f0c7c-e79f-400b-aedc-bdd56a517d67 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:51:58.843555 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:51:58.843555 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:51:58.843555 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:51:58.843555 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:51:58.843555 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:51:58.843555 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:51:58.843555 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:51:58.843555 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:51:58.843555 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:51:58.843555 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:51:58.843555 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:51:58.843555 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:51:58.843555 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:51:58.843555 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:51:58.843555 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:51:58.844669 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:51:58.844669 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:51:58.844669 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:51:58.844669 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:51:58.844669 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:51:58.844669 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:51:58.844669 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:51:58.844669 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:51:58.844669 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:51:58.844669 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:51:58.844669 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:51:58.844669 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:51:58.844669 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:51:58.844669 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:51:58.844669 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:51:58.844669 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:51:58.845702 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:51:58.845702 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:51:58.845702 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:51:58.845702 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:51:58.845702 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:51:58.845702 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:51:58.845702 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:51:58.845702 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:51:58.845702 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:51:58.845702 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:51:58.845702 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:51:58.845702 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:51:58.845702 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 585/1180] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jun 7 21:51:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:51:58.851194 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5657810-f6d4-47c4-a852-3a41873eec3c None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:58.883473 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:58.948609 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5657810-f6d4-47c4-a852-3a41873eec3c None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:58.981315 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17136fb1-1e11-46e4-8b49-a3e8d4ccfee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:58.981966 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17136fb1-1e11-46e4-8b49-a3e8d4ccfee1 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:58.982249 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17136fb1-1e11-46e4-8b49-a3e8d4ccfee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:58.982526 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17136fb1-1e11-46e4-8b49-a3e8d4ccfee1 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:58.983479 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17136fb1-1e11-46e4-8b49-a3e8d4ccfee1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:51:58.983784 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17136fb1-1e11-46e4-8b49-a3e8d4ccfee1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:58.985141 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17136fb1-1e11-46e4-8b49-a3e8d4ccfee1 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:58.992903 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 596/1181] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:51:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:58.998527 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f0c7c-e79f-400b-aedc-bdd56a517d67 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:59.094739 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f0c7c-e79f-400b-aedc-bdd56a517d67 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:59.128095 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f40a8c7-3d4f-4369-97b6-1298c2e36e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:59.128929 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f40a8c7-3d4f-4369-97b6-1298c2e36e96 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:59.129207 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f40a8c7-3d4f-4369-97b6-1298c2e36e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:59.129480 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f40a8c7-3d4f-4369-97b6-1298c2e36e96 None admin] PATH_INFO: `/v3/users/2ec3da9f21c4475a94bc273365a33ddb` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:59.141864 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f40a8c7-3d4f-4369-97b6-1298c2e36e96 None admin] RBAC: Authorizing `identity:get_user(user_id=2ec3da9f21c4475a94bc273365a33ddb)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:59.143292 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f40a8c7-3d4f-4369-97b6-1298c2e36e96 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:59.144924 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 586/1182] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:51:58 2022] GET /identity/v3/users/2ec3da9f21c4475a94bc273365a33ddb => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:59.150491 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17136fb1-1e11-46e4-8b49-a3e8d4ccfee1 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:59.243296 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:59.250529 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17136fb1-1e11-46e4-8b49-a3e8d4ccfee1 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:59.282766 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d71d54-00c3-4556-a0d4-b970cc32c297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:59.283427 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d71d54-00c3-4556-a0d4-b970cc32c297 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:59.283708 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d71d54-00c3-4556-a0d4-b970cc32c297 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:59.283980 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d71d54-00c3-4556-a0d4-b970cc32c297 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:59.349514 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d71d54-00c3-4556-a0d4-b970cc32c297 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:59.350891 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d71d54-00c3-4556-a0d4-b970cc32c297 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:59.440578 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:59.447161 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 597/1183] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:51:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 21:51:59.453035 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f40a8c7-3d4f-4369-97b6-1298c2e36e96 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:59.512873 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:51:59.575115 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f40a8c7-3d4f-4369-97b6-1298c2e36e96 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:59.608890 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb78e1-c11f-4ba5-88ab-c1dd76072749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:59.609670 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb78e1-c11f-4ba5-88ab-c1dd76072749 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:59.609949 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb78e1-c11f-4ba5-88ab-c1dd76072749 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:59.610222 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb78e1-c11f-4ba5-88ab-c1dd76072749 None admin] PATH_INFO: `/v3/projects/64f6de3d78dd4971a1fc96d2b737f186` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:59.611419 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eb78e1-c11f-4ba5-88ab-c1dd76072749 None admin] RBAC: Authorizing `identity:get_project(project_id=64f6de3d78dd4971a1fc96d2b737f186)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:59.612852 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eb78e1-c11f-4ba5-88ab-c1dd76072749 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:59.614526 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 587/1184] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:51:59 2022] GET /identity/v3/projects/64f6de3d78dd4971a1fc96d2b737f186 => generated 361 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:51:59.620681 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d71d54-00c3-4556-a0d4-b970cc32c297 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:51:59.716095 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d71d54-00c3-4556-a0d4-b970cc32c297 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:51:59.751842 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c698c7-d387-4e17-8ccb-1970a74e5226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:51:59.752517 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c698c7-d387-4e17-8ccb-1970a74e5226 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:51:59.752801 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c698c7-d387-4e17-8ccb-1970a74e5226 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:51:59.753103 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c698c7-d387-4e17-8ccb-1970a74e5226 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:51:59.753939 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c698c7-d387-4e17-8ccb-1970a74e5226 None admin] RBAC: Adding query filter params (role.id=6b483758dbd44eb4a78df2ce4603d601, scope.project.id=64f6de3d78dd4971a1fc96d2b737f186, user.id=2ec3da9f21c4475a94bc273365a33ddb) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:51:59.754252 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c698c7-d387-4e17-8ccb-1970a74e5226 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:51:59.755722 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c698c7-d387-4e17-8ccb-1970a74e5226 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:51:59.761391 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 598/1185] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:51:59 2022] GET /identity/v3/role_assignments?user.id=2ec3da9f21c4475a94bc273365a33ddb&scope.project.id=64f6de3d78dd4971a1fc96d2b737f186&role.id=6b483758dbd44eb4a78df2ce4603d601 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:52:01.961106 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c26249-4c9d-4628-aa29-57bbc0741c4d None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:52:01.961655 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c26249-4c9d-4628-aa29-57bbc0741c4d None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:52:01.961655 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c26249-4c9d-4628-aa29-57bbc0741c4d None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:52:01.963027 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 588/1186] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:52:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:52:01.968821 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c7f60-ed9f-4122-bb8f-9be0ae67c393 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:52:01.969239 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c7f60-ed9f-4122-bb8f-9be0ae67c393 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:52:01.969372 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c7f60-ed9f-4122-bb8f-9be0ae67c393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:52:01.986548 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-218c7f60-ed9f-4122-bb8f-9be0ae67c393 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:52:01.987739 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-218c7f60-ed9f-4122-bb8f-9be0ae67c393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:52:02.040297 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:52:02.051855 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 599/1187] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:52:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:52:02.073769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c26249-4c9d-4628-aa29-57bbc0741c4d None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:52:02.175047 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c26249-4c9d-4628-aa29-57bbc0741c4d None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:52:02.187730 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:52:02.208950 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb20f6-81f2-4f6f-bae1-a75bfacbd233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:52:02.209728 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb20f6-81f2-4f6f-bae1-a75bfacbd233 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:52:02.210005 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb20f6-81f2-4f6f-bae1-a75bfacbd233 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:52:02.210277 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb20f6-81f2-4f6f-bae1-a75bfacbd233 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:52:02.220638 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cb20f6-81f2-4f6f-bae1-a75bfacbd233 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:52:02.228331 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-59cb20f6-81f2-4f6f-bae1-a75bfacbd233 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:52:02.228331 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:52:02.228331 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:52:02.228331 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:52:02.228331 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:52:02.228331 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:52:02.228331 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:52:02.228331 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:52:02.228331 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:52:02.228331 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:52:02.228331 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:52:02.228331 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:52:02.228331 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:52:02.228331 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:52:02.228331 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:52:02.228331 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:52:02.229419 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:52:02.229419 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:52:02.229419 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:52:02.229419 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:52:02.229419 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:52:02.229419 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:52:02.229419 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:52:02.229419 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:52:02.229419 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:52:02.229419 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:52:02.229419 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:52:02.229419 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:52:02.229419 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:52:02.229419 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:52:02.229419 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:52:02.229419 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:52:02.230629 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:52:02.230629 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:52:02.230629 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:52:02.230629 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:52:02.230629 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:52:02.230629 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:52:02.230629 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:52:02.230629 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:52:02.230629 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:52:02.230629 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:52:02.230629 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:52:02.230629 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:52:02.230629 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 589/1188] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jun 7 21:52:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:52:02.235719 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c7f60-ed9f-4122-bb8f-9be0ae67c393 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:52:02.330680 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c7f60-ed9f-4122-bb8f-9be0ae67c393 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:52:02.363744 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd21176-f38a-47c1-95fc-62150668ecd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:52:02.364393 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd21176-f38a-47c1-95fc-62150668ecd3 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:52:02.364672 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd21176-f38a-47c1-95fc-62150668ecd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:52:02.364949 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd21176-f38a-47c1-95fc-62150668ecd3 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:52:02.365686 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd21176-f38a-47c1-95fc-62150668ecd3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:52:02.365992 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd21176-f38a-47c1-95fc-62150668ecd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:52:02.367381 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd21176-f38a-47c1-95fc-62150668ecd3 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:52:02.375178 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 600/1189] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:52:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:52:02.380898 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb20f6-81f2-4f6f-bae1-a75bfacbd233 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:52:02.477461 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb20f6-81f2-4f6f-bae1-a75bfacbd233 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:52:02.509410 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e2823-0433-4c0c-9125-c0c6b58c04b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:52:02.510251 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e2823-0433-4c0c-9125-c0c6b58c04b1 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:52:02.510531 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e2823-0433-4c0c-9125-c0c6b58c04b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:52:02.510805 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e2823-0433-4c0c-9125-c0c6b58c04b1 None admin] PATH_INFO: `/v3/users/2ec3da9f21c4475a94bc273365a33ddb` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:52:02.511964 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155e2823-0433-4c0c-9125-c0c6b58c04b1 None admin] RBAC: Authorizing `identity:get_user(user_id=2ec3da9f21c4475a94bc273365a33ddb)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:52:02.513376 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155e2823-0433-4c0c-9125-c0c6b58c04b1 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:52:02.515164 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 590/1190] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:52:02 2022] GET /identity/v3/users/2ec3da9f21c4475a94bc273365a33ddb => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:52:02.520944 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd21176-f38a-47c1-95fc-62150668ecd3 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:52:02.572448 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:52:02.619695 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd21176-f38a-47c1-95fc-62150668ecd3 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:52:02.651639 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff28f96-a701-480d-99e1-976a9632a4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:52:02.652486 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff28f96-a701-480d-99e1-976a9632a4f9 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:52:02.652763 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff28f96-a701-480d-99e1-976a9632a4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:52:02.653032 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff28f96-a701-480d-99e1-976a9632a4f9 None admin] PATH_INFO: `/v3/projects/64f6de3d78dd4971a1fc96d2b737f186` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:52:02.654191 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff28f96-a701-480d-99e1-976a9632a4f9 None admin] RBAC: Authorizing `identity:get_project(project_id=64f6de3d78dd4971a1fc96d2b737f186)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:52:02.656157 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff28f96-a701-480d-99e1-976a9632a4f9 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:52:02.657805 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 601/1191] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:52:02 2022] GET /identity/v3/projects/64f6de3d78dd4971a1fc96d2b737f186 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:52:02.663621 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e2823-0433-4c0c-9125-c0c6b58c04b1 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:52:02.704806 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:52:02.761532 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e2823-0433-4c0c-9125-c0c6b58c04b1 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:52:02.793750 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b68b6f2-5b58-43e9-9c48-d9ff0145a170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:52:02.794625 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b68b6f2-5b58-43e9-9c48-d9ff0145a170 None admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:52:02.794903 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b68b6f2-5b58-43e9-9c48-d9ff0145a170 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:52:02.795177 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b68b6f2-5b58-43e9-9c48-d9ff0145a170 None admin] PATH_INFO: `/v3/projects/64f6de3d78dd4971a1fc96d2b737f186/users/2ec3da9f21c4475a94bc273365a33ddb/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:52:02.797739 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b68b6f2-5b58-43e9-9c48-d9ff0145a170 None admin] RBAC: Authorizing `identity:create_grant(project_id=64f6de3d78dd4971a1fc96d2b737f186, user_id=2ec3da9f21c4475a94bc273365a33ddb, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:52:02.799149 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b68b6f2-5b58-43e9-9c48-d9ff0145a170 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:52:02.811659 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 591/1192] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jun 7 21:52:02 2022] PUT /identity/v3/projects/64f6de3d78dd4971a1fc96d2b737f186/users/2ec3da9f21c4475a94bc273365a33ddb/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 21:52:05.072823 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c2cdb6-1bc3-4323-af7e-fd01f82d23ae None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:52:05.073375 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c2cdb6-1bc3-4323-af7e-fd01f82d23ae None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:52:05.073375 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c2cdb6-1bc3-4323-af7e-fd01f82d23ae None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:52:05.074727 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 602/1193] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:52:05.080501 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb993657-4cba-4b10-92eb-eb9b6a592b1f None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:52:05.080922 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb993657-4cba-4b10-92eb-eb9b6a592b1f None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:52:05.081067 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb993657-4cba-4b10-92eb-eb9b6a592b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:52:05.100820 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-eb993657-4cba-4b10-92eb-eb9b6a592b1f None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:52:05.102021 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-eb993657-4cba-4b10-92eb-eb9b6a592b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:52:05.180329 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 592/1194] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:52:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 21:52:05.204795 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c2cdb6-1bc3-4323-af7e-fd01f82d23ae None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:52:05.302848 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c2cdb6-1bc3-4323-af7e-fd01f82d23ae None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:52:05.321896 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:52:05.336103 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e3eb30-c0e3-4fe9-90b2-61212eb7df33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:52:05.336824 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e3eb30-c0e3-4fe9-90b2-61212eb7df33 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:52:05.337102 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e3eb30-c0e3-4fe9-90b2-61212eb7df33 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:52:05.337373 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e3eb30-c0e3-4fe9-90b2-61212eb7df33 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:52:05.348062 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e3eb30-c0e3-4fe9-90b2-61212eb7df33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:52:05.355553 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-68e3eb30-c0e3-4fe9-90b2-61212eb7df33 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:52:05.355553 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:52:05.355553 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:52:05.355553 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:52:05.355553 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:52:05.355553 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:52:05.355553 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:52:05.355553 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:52:05.355553 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:52:05.355553 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:52:05.355553 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:52:05.355553 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:52:05.355553 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:52:05.355553 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:52:05.355553 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:52:05.355553 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:52:05.356648 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:52:05.356648 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:52:05.356648 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:52:05.356648 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:52:05.356648 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:52:05.356648 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:52:05.356648 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:52:05.356648 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:52:05.356648 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:52:05.356648 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:52:05.356648 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:52:05.356648 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:52:05.356648 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:52:05.356648 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:52:05.356648 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:52:05.356648 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:52:05.357985 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:52:05.357985 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:52:05.357985 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:52:05.357985 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:52:05.357985 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:52:05.357985 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:52:05.357985 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:52:05.357985 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:52:05.357985 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:52:05.357985 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:52:05.357985 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:52:05.357985 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:52:05.357985 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 603/1195] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jun 7 21:52:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:52:05.363042 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb993657-4cba-4b10-92eb-eb9b6a592b1f None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:52:05.393083 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:52:05.464487 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb993657-4cba-4b10-92eb-eb9b6a592b1f None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:52:05.500020 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aadca62-855c-47a6-bd7e-963d2571e08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:52:05.500685 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aadca62-855c-47a6-bd7e-963d2571e08a None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:52:05.500966 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aadca62-855c-47a6-bd7e-963d2571e08a None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:52:05.501241 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aadca62-855c-47a6-bd7e-963d2571e08a None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:52:05.501982 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aadca62-855c-47a6-bd7e-963d2571e08a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:52:05.502291 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aadca62-855c-47a6-bd7e-963d2571e08a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:52:05.503657 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aadca62-855c-47a6-bd7e-963d2571e08a None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:52:05.511420 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 593/1196] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:52:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:52:05.517953 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e3eb30-c0e3-4fe9-90b2-61212eb7df33 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:52:05.615813 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e3eb30-c0e3-4fe9-90b2-61212eb7df33 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:52:05.648587 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca28690-ed87-42d7-8b95-6bbc12c762a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:52:05.649436 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca28690-ed87-42d7-8b95-6bbc12c762a0 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:52:05.649714 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca28690-ed87-42d7-8b95-6bbc12c762a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:52:05.649983 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca28690-ed87-42d7-8b95-6bbc12c762a0 None admin] PATH_INFO: `/v3/users/2ec3da9f21c4475a94bc273365a33ddb` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:52:05.651139 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca28690-ed87-42d7-8b95-6bbc12c762a0 None admin] RBAC: Authorizing `identity:get_user(user_id=2ec3da9f21c4475a94bc273365a33ddb)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:52:05.652596 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca28690-ed87-42d7-8b95-6bbc12c762a0 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:52:05.654178 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 604/1197] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jun 7 21:52:05 2022] GET /identity/v3/users/2ec3da9f21c4475a94bc273365a33ddb => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:52:05.660221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aadca62-855c-47a6-bd7e-963d2571e08a None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:52:05.750694 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:52:05.760155 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aadca62-855c-47a6-bd7e-963d2571e08a None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:52:05.793712 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e736b1a6-902d-4c37-bb32-a3db6dfcccb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:52:05.794356 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e736b1a6-902d-4c37-bb32-a3db6dfcccb0 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:52:05.794640 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e736b1a6-902d-4c37-bb32-a3db6dfcccb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:52:05.794916 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e736b1a6-902d-4c37-bb32-a3db6dfcccb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:52:05.867911 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e736b1a6-902d-4c37-bb32-a3db6dfcccb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:52:05.869281 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e736b1a6-902d-4c37-bb32-a3db6dfcccb0 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:52:05.961855 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:52:05.972055 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 594/1198] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:52:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 21:52:05.978103 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca28690-ed87-42d7-8b95-6bbc12c762a0 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:52:06.033104 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:52:06.084108 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca28690-ed87-42d7-8b95-6bbc12c762a0 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:52:06.117498 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae19bb8-0d62-4e61-9455-7a7d88476ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:52:06.118389 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae19bb8-0d62-4e61-9455-7a7d88476ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:52:06.118616 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae19bb8-0d62-4e61-9455-7a7d88476ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:52:06.118886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae19bb8-0d62-4e61-9455-7a7d88476ad8 None admin] PATH_INFO: `/v3/projects/64f6de3d78dd4971a1fc96d2b737f186` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:52:06.120059 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae19bb8-0d62-4e61-9455-7a7d88476ad8 None admin] RBAC: Authorizing `identity:get_project(project_id=64f6de3d78dd4971a1fc96d2b737f186)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:52:06.121477 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae19bb8-0d62-4e61-9455-7a7d88476ad8 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:52:06.123134 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 605/1199] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jun 7 21:52:05 2022] GET /identity/v3/projects/64f6de3d78dd4971a1fc96d2b737f186 => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:52:06.129907 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e736b1a6-902d-4c37-bb32-a3db6dfcccb0 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:52:06.232649 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e736b1a6-902d-4c37-bb32-a3db6dfcccb0 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:52:06.269264 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e603a1cc-a924-4a4b-98d8-488afb9fe50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:52:06.269977 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e603a1cc-a924-4a4b-98d8-488afb9fe50e None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:52:06.270302 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e603a1cc-a924-4a4b-98d8-488afb9fe50e None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:52:06.270622 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e603a1cc-a924-4a4b-98d8-488afb9fe50e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:52:06.271539 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e603a1cc-a924-4a4b-98d8-488afb9fe50e None admin] RBAC: Adding query filter params (role.id=6b483758dbd44eb4a78df2ce4603d601, scope.project.id=64f6de3d78dd4971a1fc96d2b737f186, user.id=2ec3da9f21c4475a94bc273365a33ddb) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:52:06.271896 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e603a1cc-a924-4a4b-98d8-488afb9fe50e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:52:06.273434 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e603a1cc-a924-4a4b-98d8-488afb9fe50e None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:52:06.280325 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 595/1200] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:52:06 2022] GET /identity/v3/role_assignments?user.id=2ec3da9f21c4475a94bc273365a33ddb&scope.project.id=64f6de3d78dd4971a1fc96d2b737f186&role.id=6b483758dbd44eb4a78df2ce4603d601 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:52:16.113864 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=42 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:52:16.276849 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=72 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:07.318011 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da520d1f-087e-462d-8d9f-39484d2453bb None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:07.318628 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da520d1f-087e-462d-8d9f-39484d2453bb None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:07.318628 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da520d1f-087e-462d-8d9f-39484d2453bb None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:07.320156 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 606/1201] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:53:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:53:07.325844 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eba1d8-8bad-44d0-9f44-7c14842f4d18 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:07.326325 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eba1d8-8bad-44d0-9f44-7c14842f4d18 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:07.326411 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eba1d8-8bad-44d0-9f44-7c14842f4d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:07.344896 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-b7eba1d8-8bad-44d0-9f44-7c14842f4d18 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:53:07.346087 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-b7eba1d8-8bad-44d0-9f44-7c14842f4d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:53:07.436115 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 596/1202] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:53:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:53:07.466394 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da520d1f-087e-462d-8d9f-39484d2453bb None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:07.587661 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da520d1f-087e-462d-8d9f-39484d2453bb None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:07.628698 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6acfc2-ab4e-45a0-9cf2-a1e0e6a0c3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:07.629787 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6acfc2-ab4e-45a0-9cf2-a1e0e6a0c3fe None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:07.630229 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6acfc2-ab4e-45a0-9cf2-a1e0e6a0c3fe None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:07.630665 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6acfc2-ab4e-45a0-9cf2-a1e0e6a0c3fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:07.642335 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6acfc2-ab4e-45a0-9cf2-a1e0e6a0c3fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:07.643798 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6acfc2-ab4e-45a0-9cf2-a1e0e6a0c3fe None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:07.662600 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-bd6acfc2-ab4e-45a0-9cf2-a1e0e6a0c3fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:53:07.662600 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:53:07.662600 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:53:07.662600 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:53:07.662600 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:53:07.662600 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:53:07.662600 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:53:07.662600 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:53:07.662600 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:53:07.662600 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:53:07.662600 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:53:07.662600 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:53:07.662600 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:53:07.662600 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:53:07.662600 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:53:07.664006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:53:07.664006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:53:07.664006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:53:07.664006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:53:07.664006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:53:07.664006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:53:07.664006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:53:07.664006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:53:07.664006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:53:07.664006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:53:07.664006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:53:07.664006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:53:07.664006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:53:07.664006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:53:07.664006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:53:07.664006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:53:07.664006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:53:07.665014 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:53:07.665014 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:53:07.665014 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:53:07.665014 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:53:07.669446 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 607/1203] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:53:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:53:07.671065 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eba1d8-8bad-44d0-9f44-7c14842f4d18 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:07.812570 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eba1d8-8bad-44d0-9f44-7c14842f4d18 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:07.859944 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:07.866929 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a62748-f3d9-4af9-aa5a-2cd53de28b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:07.867867 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a62748-f3d9-4af9-aa5a-2cd53de28b82 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:07.868159 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a62748-f3d9-4af9-aa5a-2cd53de28b82 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:07.868567 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a62748-f3d9-4af9-aa5a-2cd53de28b82 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:07.869274 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a62748-f3d9-4af9-aa5a-2cd53de28b82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:53:07.869596 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a62748-f3d9-4af9-aa5a-2cd53de28b82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:07.871127 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a62748-f3d9-4af9-aa5a-2cd53de28b82 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:07.891364 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 597/1204] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:53:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:53:07.894425 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6acfc2-ab4e-45a0-9cf2-a1e0e6a0c3fe None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:07.966899 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:08.021035 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6acfc2-ab4e-45a0-9cf2-a1e0e6a0c3fe None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:08.068844 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f14ac5-2323-45b5-9646-733536c67543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:08.069958 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f14ac5-2323-45b5-9646-733536c67543 None admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:08.070469 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f14ac5-2323-45b5-9646-733536c67543 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:08.071053 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f14ac5-2323-45b5-9646-733536c67543 None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:08.072331 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f14ac5-2323-45b5-9646-733536c67543 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:08.074420 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f14ac5-2323-45b5-9646-733536c67543 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:08.098832 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 608/1205] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Jun 7 21:53:07 2022] POST /identity/v3/users => generated 251 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 21:53:10.550043 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde2fb3-ddae-4eba-84a8-4a8e26744f47 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:10.550701 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde2fb3-ddae-4eba-84a8-4a8e26744f47 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:10.550701 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde2fb3-ddae-4eba-84a8-4a8e26744f47 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:10.551998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 598/1206] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:53:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 21:53:10.557421 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0b47c-c22d-4f11-b679-47b3e3fb8297 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:10.557899 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0b47c-c22d-4f11-b679-47b3e3fb8297 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:10.557997 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0b47c-c22d-4f11-b679-47b3e3fb8297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:10.576352 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-60b0b47c-c22d-4f11-b679-47b3e3fb8297 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:53:10.577536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-60b0b47c-c22d-4f11-b679-47b3e3fb8297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:53:10.647943 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 609/1207] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:53:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:53:10.672405 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde2fb3-ddae-4eba-84a8-4a8e26744f47 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:10.807106 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde2fb3-ddae-4eba-84a8-4a8e26744f47 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:10.844037 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a785244b-f073-4be3-9b94-ad433a09201e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:10.844908 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a785244b-f073-4be3-9b94-ad433a09201e None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:10.845193 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a785244b-f073-4be3-9b94-ad433a09201e None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:10.845464 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a785244b-f073-4be3-9b94-ad433a09201e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:10.857247 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a785244b-f073-4be3-9b94-ad433a09201e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:53:10.864930 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-a785244b-f073-4be3-9b94-ad433a09201e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 21:53:10.864930 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:53:10.864930 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:53:10.864930 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:53:10.864930 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:53:10.864930 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:53:10.864930 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:53:10.864930 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:53:10.864930 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:53:10.864930 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:53:10.864930 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:53:10.864930 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:53:10.864930 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:53:10.864930 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:53:10.864930 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:53:10.864930 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:53:10.866221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:53:10.866221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:53:10.866221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:53:10.866221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:53:10.866221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:53:10.866221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:53:10.866221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:53:10.866221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:53:10.866221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:53:10.866221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:53:10.866221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:53:10.866221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:53:10.866221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:53:10.866221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:53:10.866221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:53:10.866221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:53:10.867091 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:53:10.867091 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:53:10.867091 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:53:10.867091 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:53:10.867091 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:53:10.867091 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:53:10.867091 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:53:10.867091 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:53:10.867091 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:53:10.867091 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:53:10.867091 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 21:53:10.867091 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:53:10.867091 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 599/1208] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jun 7 21:53:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:53:10.872680 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b0b47c-c22d-4f11-b679-47b3e3fb8297 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:10.919400 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:11.001986 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b0b47c-c22d-4f11-b679-47b3e3fb8297 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:11.041460 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ef5ea4-877e-4e54-81ff-a80456878720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:11.042981 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef5ea4-877e-4e54-81ff-a80456878720 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:11.042981 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef5ea4-877e-4e54-81ff-a80456878720 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:11.042981 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef5ea4-877e-4e54-81ff-a80456878720 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:11.043571 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ef5ea4-877e-4e54-81ff-a80456878720 None admin] RBAC: Adding query filter params (name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:53:11.044115 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ef5ea4-877e-4e54-81ff-a80456878720 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:11.045499 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ef5ea4-877e-4e54-81ff-a80456878720 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:11.054261 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 610/1209] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Jun 7 21:53:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:53:11.061768 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a785244b-f073-4be3-9b94-ad433a09201e None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:11.124093 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:11.181543 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a785244b-f073-4be3-9b94-ad433a09201e None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:11.222073 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf06392-99ad-4cfd-96c0-3dfac5ad2f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:11.223207 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf06392-99ad-4cfd-96c0-3dfac5ad2f74 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:11.223521 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf06392-99ad-4cfd-96c0-3dfac5ad2f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:11.223831 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf06392-99ad-4cfd-96c0-3dfac5ad2f74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:11.236430 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf06392-99ad-4cfd-96c0-3dfac5ad2f74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:11.238548 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf06392-99ad-4cfd-96c0-3dfac5ad2f74 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:11.258181 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-fbf06392-99ad-4cfd-96c0-3dfac5ad2f74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:53:11.258181 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:53:11.258181 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:53:11.258181 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:53:11.258181 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:53:11.258181 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:53:11.258181 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:53:11.258181 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:53:11.258181 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:53:11.258181 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:53:11.258181 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:53:11.258181 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:53:11.258181 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:53:11.258181 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:53:11.258181 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:53:11.259205 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:53:11.259205 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:53:11.259205 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:53:11.259205 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:53:11.259205 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:53:11.259205 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:53:11.259205 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:53:11.259205 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:53:11.259205 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:53:11.259205 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:53:11.259205 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:53:11.259205 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:53:11.259205 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:53:11.259205 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:53:11.259205 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:53:11.259205 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:53:11.259205 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:53:11.260157 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:53:11.260157 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:53:11.260157 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:53:11.260157 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:53:11.260157 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 600/1210] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:53:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:53:11.265875 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ef5ea4-877e-4e54-81ff-a80456878720 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:11.395889 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:11.399094 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ef5ea4-877e-4e54-81ff-a80456878720 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:11.439885 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89c7952-2c30-4b1b-99b3-2b0e421994df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:11.440655 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c7952-2c30-4b1b-99b3-2b0e421994df None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:11.441017 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c7952-2c30-4b1b-99b3-2b0e421994df None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:11.441366 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c7952-2c30-4b1b-99b3-2b0e421994df None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:11.442163 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89c7952-2c30-4b1b-99b3-2b0e421994df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:53:11.442547 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89c7952-2c30-4b1b-99b3-2b0e421994df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:11.444232 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89c7952-2c30-4b1b-99b3-2b0e421994df None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:11.458729 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 611/1211] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:53:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:53:11.467283 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf06392-99ad-4cfd-96c0-3dfac5ad2f74 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:11.599858 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf06392-99ad-4cfd-96c0-3dfac5ad2f74 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:11.601851 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:11.643702 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c78599f-fcb3-4978-90e2-9eb2693ccf2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:11.644607 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c78599f-fcb3-4978-90e2-9eb2693ccf2b None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:11.644921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c78599f-fcb3-4978-90e2-9eb2693ccf2b None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:11.645194 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c78599f-fcb3-4978-90e2-9eb2693ccf2b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:11.657002 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c78599f-fcb3-4978-90e2-9eb2693ccf2b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:11.658500 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c78599f-fcb3-4978-90e2-9eb2693ccf2b None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:11.677464 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-8c78599f-fcb3-4978-90e2-9eb2693ccf2b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 21:53:11.677464 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:53:11.677464 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:53:11.677464 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:53:11.677464 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:53:11.677464 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:53:11.677464 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:53:11.677464 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:53:11.677464 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:53:11.677464 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:53:11.677464 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:53:11.677464 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:53:11.677464 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 21:53:11.677464 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 21:53:11.677464 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 21:53:11.677464 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 21:53:11.678724 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:53:11.678724 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:53:11.678724 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 21:53:11.678724 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:53:11.678724 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 21:53:11.678724 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:53:11.678724 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:53:11.678724 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:53:11.678724 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:53:11.678724 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:53:11.678724 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:53:11.678724 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:53:11.678724 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:53:11.678724 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:53:11.678724 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:53:11.678724 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:53:11.678724 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:53:11.678724 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:53:11.685112 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:53:11.685112 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:53:11.685112 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 21:53:11.685112 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 21:53:11.685112 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 21:53:11.685112 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 21:53:11.685112 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 21:53:11.685112 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 21:53:11.685112 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 21:53:11.685112 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:53:11.685112 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 601/1212] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:53:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:53:11.689242 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89c7952-2c30-4b1b-99b3-2b0e421994df None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:11.820563 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89c7952-2c30-4b1b-99b3-2b0e421994df None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:11.862210 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635718a9-c3a5-495f-b619-a1645ada5afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:11.863199 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635718a9-c3a5-495f-b619-a1645ada5afb None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:11.863607 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635718a9-c3a5-495f-b619-a1645ada5afb None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:11.863932 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635718a9-c3a5-495f-b619-a1645ada5afb None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:11.865141 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635718a9-c3a5-495f-b619-a1645ada5afb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:53:11.865461 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635718a9-c3a5-495f-b619-a1645ada5afb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:11.867183 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635718a9-c3a5-495f-b619-a1645ada5afb None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:11.891399 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 612/1213] 10.222.0.13 () {56 vars in 1143 bytes} [Tue Jun 7 21:53:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:53:11.897376 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c78599f-fcb3-4978-90e2-9eb2693ccf2b None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:12.024703 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c78599f-fcb3-4978-90e2-9eb2693ccf2b None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:12.066257 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b93f9-691c-4c2e-96a2-b1ce8f707751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:12.066982 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b93f9-691c-4c2e-96a2-b1ce8f707751 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:12.067290 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b93f9-691c-4c2e-96a2-b1ce8f707751 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:12.067598 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b93f9-691c-4c2e-96a2-b1ce8f707751 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:12.107103 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:12.174777 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6b93f9-691c-4c2e-96a2-b1ce8f707751 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:12.176653 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6b93f9-691c-4c2e-96a2-b1ce8f707751 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:12.297140 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 602/1214] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:53:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 21:53:12.298677 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635718a9-c3a5-495f-b619-a1645ada5afb None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:12.318707 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:12.405673 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635718a9-c3a5-495f-b619-a1645ada5afb None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:12.441158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b255569c-1215-464f-9041-afec04004736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:12.442557 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b255569c-1215-464f-9041-afec04004736 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:12.442827 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b255569c-1215-464f-9041-afec04004736 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:12.443127 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b255569c-1215-464f-9041-afec04004736 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:12.455434 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b255569c-1215-464f-9041-afec04004736 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:12.457007 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b255569c-1215-464f-9041-afec04004736 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:12.476177 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-b255569c-1215-464f-9041-afec04004736 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:53:12.476177 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:53:12.476177 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:53:12.476177 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:53:12.476177 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:53:12.476177 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:53:12.476177 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:53:12.476177 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:53:12.476177 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:53:12.476177 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:53:12.476177 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:53:12.476177 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:53:12.476177 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:53:12.476177 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:53:12.476177 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:53:12.477570 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:53:12.477570 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:53:12.477570 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:53:12.477570 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:53:12.477570 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:53:12.477570 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:53:12.477570 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:53:12.477570 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:53:12.477570 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:53:12.477570 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:53:12.477570 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:53:12.477570 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:53:12.477570 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:53:12.477570 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:53:12.477570 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:53:12.477570 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:53:12.477570 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:53:12.483530 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:53:12.483530 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:53:12.483530 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:53:12.483530 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:53:12.483530 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 613/1215] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:53:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:53:12.485669 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b93f9-691c-4c2e-96a2-b1ce8f707751 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:12.554616 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:12.597017 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b93f9-691c-4c2e-96a2-b1ce8f707751 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:12.635103 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65380753-456f-4265-86bf-a7522b0dc958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:12.637478 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65380753-456f-4265-86bf-a7522b0dc958 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:12.637478 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65380753-456f-4265-86bf-a7522b0dc958 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:12.637478 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65380753-456f-4265-86bf-a7522b0dc958 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:12.637836 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65380753-456f-4265-86bf-a7522b0dc958 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:53:12.638084 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65380753-456f-4265-86bf-a7522b0dc958 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:12.639499 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65380753-456f-4265-86bf-a7522b0dc958 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:12.652191 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 603/1216] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:53:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:53:12.658346 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b255569c-1215-464f-9041-afec04004736 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:12.732039 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:12.779563 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b255569c-1215-464f-9041-afec04004736 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:12.817753 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e41d71-2f18-445c-a272-650663bdaa0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:12.818607 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e41d71-2f18-445c-a272-650663bdaa0f None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:12.818988 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e41d71-2f18-445c-a272-650663bdaa0f None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:12.819384 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e41d71-2f18-445c-a272-650663bdaa0f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:12.826805 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e41d71-2f18-445c-a272-650663bdaa0f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:12.828393 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e41d71-2f18-445c-a272-650663bdaa0f None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:12.842240 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-53e41d71-2f18-445c-a272-650663bdaa0f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:53:12.842240 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:53:12.842240 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:53:12.842240 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:53:12.842240 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:53:12.842240 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:53:12.842240 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:53:12.842240 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:53:12.842240 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:53:12.842240 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:53:12.842240 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:53:12.842240 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:53:12.842240 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 21:53:12.842240 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 21:53:12.842240 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 21:53:12.843440 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 21:53:12.843440 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:53:12.843440 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:53:12.843440 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:53:12.843440 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:53:12.843440 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:53:12.843440 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:53:12.843440 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:53:12.843440 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:53:12.843440 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:53:12.843440 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:53:12.843440 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:53:12.843440 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:53:12.843440 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:53:12.843440 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:53:12.843440 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:53:12.843440 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:53:12.844556 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 21:53:12.844556 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 21:53:12.844556 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 21:53:12.844556 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 21:53:12.844556 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 21:53:12.844556 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 21:53:12.844556 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:53:12.844556 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:53:12.846097 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 614/1217] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:53:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:53:12.852083 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65380753-456f-4265-86bf-a7522b0dc958 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:12.963424 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65380753-456f-4265-86bf-a7522b0dc958 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:12.990226 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:13.003317 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4837cf18-cc6b-4269-b15f-dab560b69b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:13.004014 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4837cf18-cc6b-4269-b15f-dab560b69b74 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:13.004293 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4837cf18-cc6b-4269-b15f-dab560b69b74 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:13.004569 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4837cf18-cc6b-4269-b15f-dab560b69b74 None admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:13.005378 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4837cf18-cc6b-4269-b15f-dab560b69b74 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:53:13.005702 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4837cf18-cc6b-4269-b15f-dab560b69b74 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:13.007120 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4837cf18-cc6b-4269-b15f-dab560b69b74 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:13.025713 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 604/1218] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jun 7 21:53:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:53:13.032044 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e41d71-2f18-445c-a272-650663bdaa0f None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:13.158414 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e41d71-2f18-445c-a272-650663bdaa0f None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:13.195303 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:13.202209 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c01e87-5e69-492f-a09c-9b7e28ac58d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:13.202872 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c01e87-5e69-492f-a09c-9b7e28ac58d7 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:13.203152 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c01e87-5e69-492f-a09c-9b7e28ac58d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:13.203447 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c01e87-5e69-492f-a09c-9b7e28ac58d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:13.204336 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c01e87-5e69-492f-a09c-9b7e28ac58d7 None admin] RBAC: Adding query filter params (role.id=2c7f73c184d0406bbf2899465953342b, scope.project.id=0219dbda54274fdc9bdccbd960c30c6b, user.id=02941c6ad2d54e48b1114d9e030c66c7) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:53:13.204608 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c01e87-5e69-492f-a09c-9b7e28ac58d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:13.206017 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c01e87-5e69-492f-a09c-9b7e28ac58d7 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:13.219589 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 615/1219] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:53:13 2022] GET /identity/v3/role_assignments?user.id=02941c6ad2d54e48b1114d9e030c66c7&scope.project.id=0219dbda54274fdc9bdccbd960c30c6b&role.id=2c7f73c184d0406bbf2899465953342b => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:53:15.494454 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d2d55-15ac-4180-91f7-e5c0feaf1192 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:15.495031 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d2d55-15ac-4180-91f7-e5c0feaf1192 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:15.495031 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d2d55-15ac-4180-91f7-e5c0feaf1192 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:15.499389 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 605/1220] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:53:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:53:15.502580 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9caf3d9-35d0-4dbf-8cdc-24fb469f2ea3 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:15.503405 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9caf3d9-35d0-4dbf-8cdc-24fb469f2ea3 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:15.503905 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9caf3d9-35d0-4dbf-8cdc-24fb469f2ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:15.522673 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-c9caf3d9-35d0-4dbf-8cdc-24fb469f2ea3 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:53:15.524207 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c9caf3d9-35d0-4dbf-8cdc-24fb469f2ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:53:15.594825 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 616/1221] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:53:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:53:15.618561 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d2d55-15ac-4180-91f7-e5c0feaf1192 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:15.729236 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d2d55-15ac-4180-91f7-e5c0feaf1192 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:15.770440 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27ad497-ae76-4faa-81d9-ec933a0c12cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:15.771296 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27ad497-ae76-4faa-81d9-ec933a0c12cd None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:15.771599 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27ad497-ae76-4faa-81d9-ec933a0c12cd None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:15.771907 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27ad497-ae76-4faa-81d9-ec933a0c12cd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:15.788465 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27ad497-ae76-4faa-81d9-ec933a0c12cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:53:15.796841 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-f27ad497-ae76-4faa-81d9-ec933a0c12cd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 21:53:15.796841 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:53:15.796841 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:53:15.796841 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:53:15.796841 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:53:15.796841 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:53:15.796841 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:53:15.796841 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:53:15.796841 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:53:15.796841 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:53:15.796841 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:53:15.796841 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:53:15.796841 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:53:15.796841 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:53:15.796841 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:53:15.796841 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:53:15.797808 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:53:15.797808 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:53:15.797808 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:53:15.797808 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:53:15.797808 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:53:15.797808 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:53:15.797808 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:53:15.797808 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:53:15.797808 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:53:15.797808 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:53:15.797808 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:53:15.797808 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:53:15.797808 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:53:15.797808 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:53:15.797808 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:53:15.797808 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:53:15.799214 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:53:15.799214 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:53:15.799214 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:53:15.799214 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:53:15.799214 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:53:15.799214 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:53:15.799214 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:53:15.799214 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:53:15.799214 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:53:15.799214 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:53:15.799214 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 21:53:15.799214 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:53:15.799214 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 606/1222] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jun 7 21:53:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:53:15.804422 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9caf3d9-35d0-4dbf-8cdc-24fb469f2ea3 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:15.908444 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9caf3d9-35d0-4dbf-8cdc-24fb469f2ea3 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:15.932965 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:15.945592 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b2a26c-a94a-4ae8-b917-53cb1ee427b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:15.946266 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b2a26c-a94a-4ae8-b917-53cb1ee427b3 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:15.946546 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b2a26c-a94a-4ae8-b917-53cb1ee427b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:15.946819 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b2a26c-a94a-4ae8-b917-53cb1ee427b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:15.947629 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b2a26c-a94a-4ae8-b917-53cb1ee427b3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:53:15.947940 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b2a26c-a94a-4ae8-b917-53cb1ee427b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:15.949941 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b2a26c-a94a-4ae8-b917-53cb1ee427b3 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:15.960505 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 617/1223] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Jun 7 21:53:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:53:15.967003 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27ad497-ae76-4faa-81d9-ec933a0c12cd None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:16.019464 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:16.086479 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27ad497-ae76-4faa-81d9-ec933a0c12cd None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:16.128940 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8173d5-377e-4a4e-a707-a313bc4bc4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:16.129739 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8173d5-377e-4a4e-a707-a313bc4bc4e5 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:16.130022 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8173d5-377e-4a4e-a707-a313bc4bc4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:16.130319 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8173d5-377e-4a4e-a707-a313bc4bc4e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:16.141662 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8173d5-377e-4a4e-a707-a313bc4bc4e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:16.143124 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8173d5-377e-4a4e-a707-a313bc4bc4e5 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:16.162585 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-fa8173d5-377e-4a4e-a707-a313bc4bc4e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:53:16.162585 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:53:16.162585 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:53:16.162585 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:53:16.162585 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:53:16.162585 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:53:16.162585 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:53:16.162585 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:53:16.162585 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:53:16.162585 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:53:16.162585 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:53:16.162585 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:53:16.162585 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:53:16.162585 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:53:16.162585 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:53:16.163597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:53:16.163597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:53:16.163597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:53:16.163597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:53:16.163597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:53:16.163597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:53:16.163597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:53:16.163597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:53:16.163597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:53:16.163597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:53:16.163597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:53:16.163597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:53:16.163597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:53:16.163597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:53:16.163597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:53:16.163597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:53:16.163597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:53:16.164567 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:53:16.164567 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:53:16.164567 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:53:16.164567 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:53:16.164567 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 607/1224] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:53:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:53:16.170695 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b2a26c-a94a-4ae8-b917-53cb1ee427b3 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:16.280586 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b2a26c-a94a-4ae8-b917-53cb1ee427b3 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:16.317081 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d7e26-53ad-44a9-8dad-f96f06e60fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:16.317741 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d7e26-53ad-44a9-8dad-f96f06e60fc5 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:16.318018 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d7e26-53ad-44a9-8dad-f96f06e60fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:16.318290 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d7e26-53ad-44a9-8dad-f96f06e60fc5 None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:16.319028 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87d7e26-53ad-44a9-8dad-f96f06e60fc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:53:16.319313 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87d7e26-53ad-44a9-8dad-f96f06e60fc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:16.320705 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87d7e26-53ad-44a9-8dad-f96f06e60fc5 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:16.334793 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 618/1225] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:53:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:53:16.342016 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8173d5-377e-4a4e-a707-a313bc4bc4e5 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:16.443079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:16.453185 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8173d5-377e-4a4e-a707-a313bc4bc4e5 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:16.504482 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab722a2-e266-4865-91c5-eb6abe447484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:16.505346 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab722a2-e266-4865-91c5-eb6abe447484 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:16.505581 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab722a2-e266-4865-91c5-eb6abe447484 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:16.505887 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab722a2-e266-4865-91c5-eb6abe447484 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:16.517521 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab722a2-e266-4865-91c5-eb6abe447484 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:16.519033 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab722a2-e266-4865-91c5-eb6abe447484 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:16.544574 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-aab722a2-e266-4865-91c5-eb6abe447484 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 21:53:16.544574 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:53:16.544574 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:53:16.544574 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:53:16.544574 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:53:16.544574 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:53:16.544574 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:53:16.544574 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:53:16.544574 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:53:16.544574 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:53:16.544574 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:53:16.544574 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:53:16.544574 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 21:53:16.544574 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 21:53:16.544574 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 21:53:16.544574 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 21:53:16.546029 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:53:16.546029 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:53:16.546029 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 21:53:16.546029 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:53:16.546029 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 21:53:16.546029 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:53:16.546029 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:53:16.546029 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:53:16.546029 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:53:16.546029 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:53:16.546029 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:53:16.546029 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:53:16.546029 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:53:16.546029 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:53:16.546029 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:53:16.546029 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:53:16.546029 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:53:16.546029 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:53:16.552264 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:53:16.552264 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:53:16.552264 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 21:53:16.552264 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 21:53:16.552264 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 21:53:16.552264 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 21:53:16.552264 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 21:53:16.552264 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 21:53:16.552264 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 21:53:16.552264 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:53:16.552264 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 608/1226] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:53:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:53:16.554876 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d7e26-53ad-44a9-8dad-f96f06e60fc5 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:16.627362 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:16.689742 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d7e26-53ad-44a9-8dad-f96f06e60fc5 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:16.729523 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980ecbc3-89f9-40d8-bda4-59689442fa15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:16.731049 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980ecbc3-89f9-40d8-bda4-59689442fa15 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:16.731587 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980ecbc3-89f9-40d8-bda4-59689442fa15 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:16.731877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980ecbc3-89f9-40d8-bda4-59689442fa15 None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:16.733105 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980ecbc3-89f9-40d8-bda4-59689442fa15 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:53:16.733588 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980ecbc3-89f9-40d8-bda4-59689442fa15 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:16.735719 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980ecbc3-89f9-40d8-bda4-59689442fa15 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:16.756930 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 619/1227] 10.222.0.13 () {56 vars in 1143 bytes} [Tue Jun 7 21:53:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:53:16.763019 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab722a2-e266-4865-91c5-eb6abe447484 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:16.876538 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab722a2-e266-4865-91c5-eb6abe447484 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:16.912998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70dc603-50ae-42dc-b327-929ffd0a9f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:16.913793 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70dc603-50ae-42dc-b327-929ffd0a9f44 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:16.914147 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70dc603-50ae-42dc-b327-929ffd0a9f44 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:16.914440 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70dc603-50ae-42dc-b327-929ffd0a9f44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:16.926980 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70dc603-50ae-42dc-b327-929ffd0a9f44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:16.928531 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70dc603-50ae-42dc-b327-929ffd0a9f44 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:16.946237 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-c70dc603-50ae-42dc-b327-929ffd0a9f44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:53:16.946237 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:53:16.946237 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:53:16.946237 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:53:16.946237 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:53:16.946237 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:53:16.946237 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:53:16.946237 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:53:16.946237 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:53:16.946237 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:53:16.946237 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:53:16.946237 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:53:16.946237 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:53:16.946237 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:53:16.946237 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:53:16.947492 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:53:16.947492 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:53:16.947492 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:53:16.947492 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:53:16.947492 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:53:16.947492 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:53:16.947492 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:53:16.947492 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:53:16.947492 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:53:16.947492 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:53:16.947492 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:53:16.947492 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:53:16.947492 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:53:16.947492 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:53:16.947492 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:53:16.947492 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:53:16.947492 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:53:16.953458 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:53:16.953458 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:53:16.953458 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:53:16.953458 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:53:16.953458 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 609/1228] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:53:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:53:16.953766 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980ecbc3-89f9-40d8-bda4-59689442fa15 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:17.069359 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980ecbc3-89f9-40d8-bda4-59689442fa15 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:17.072229 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:17.109615 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fdf384-efc4-451d-a211-be8b8fc4f89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:17.110379 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fdf384-efc4-451d-a211-be8b8fc4f89f None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:17.110661 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fdf384-efc4-451d-a211-be8b8fc4f89f None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:17.110937 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fdf384-efc4-451d-a211-be8b8fc4f89f None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:17.111782 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fdf384-efc4-451d-a211-be8b8fc4f89f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:53:17.112090 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fdf384-efc4-451d-a211-be8b8fc4f89f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:17.113636 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fdf384-efc4-451d-a211-be8b8fc4f89f None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:17.131356 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 620/1229] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:53:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:53:17.132612 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70dc603-50ae-42dc-b327-929ffd0a9f44 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:17.142558 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:17.253723 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70dc603-50ae-42dc-b327-929ffd0a9f44 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:17.298768 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02979159-e006-4f37-9370-94030eff9963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:17.299657 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02979159-e006-4f37-9370-94030eff9963 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:17.299977 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02979159-e006-4f37-9370-94030eff9963 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:17.300226 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02979159-e006-4f37-9370-94030eff9963 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:17.308151 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02979159-e006-4f37-9370-94030eff9963 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:17.309981 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02979159-e006-4f37-9370-94030eff9963 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:17.325734 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-02979159-e006-4f37-9370-94030eff9963 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:53:17.325734 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:53:17.325734 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:53:17.325734 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:53:17.325734 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:53:17.325734 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:53:17.325734 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:53:17.325734 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:53:17.325734 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:53:17.325734 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:53:17.325734 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:53:17.325734 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:53:17.325734 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 21:53:17.325734 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 21:53:17.325734 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 21:53:17.326829 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 21:53:17.326829 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:53:17.326829 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:53:17.326829 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:53:17.326829 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:53:17.326829 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:53:17.326829 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:53:17.326829 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:53:17.326829 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:53:17.326829 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:53:17.326829 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:53:17.326829 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:53:17.326829 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:53:17.326829 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:53:17.326829 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:53:17.326829 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:53:17.326829 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:53:17.328109 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 21:53:17.328109 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 21:53:17.328109 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 21:53:17.328109 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 21:53:17.328109 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 21:53:17.328109 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 21:53:17.328109 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:53:17.328109 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:53:17.328109 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 610/1230] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:53:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:53:17.333194 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fdf384-efc4-451d-a211-be8b8fc4f89f None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:17.451510 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fdf384-efc4-451d-a211-be8b8fc4f89f None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:17.485976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90d0d45-c7ea-4a5b-8dc1-693091b1a27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:17.486635 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d0d45-c7ea-4a5b-8dc1-693091b1a27e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:17.486915 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d0d45-c7ea-4a5b-8dc1-693091b1a27e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:17.487229 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d0d45-c7ea-4a5b-8dc1-693091b1a27e None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:17.488027 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90d0d45-c7ea-4a5b-8dc1-693091b1a27e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:53:17.488301 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90d0d45-c7ea-4a5b-8dc1-693091b1a27e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:17.489730 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90d0d45-c7ea-4a5b-8dc1-693091b1a27e None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:17.502690 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 621/1231] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jun 7 21:53:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:53:17.508827 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02979159-e006-4f37-9370-94030eff9963 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:17.582940 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:17.623775 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02979159-e006-4f37-9370-94030eff9963 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:17.662180 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac24fab-6856-4a1d-87e4-400b3d34b035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:17.663036 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac24fab-6856-4a1d-87e4-400b3d34b035 None admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:17.663320 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac24fab-6856-4a1d-87e4-400b3d34b035 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:17.663617 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac24fab-6856-4a1d-87e4-400b3d34b035 None admin] PATH_INFO: `/v3/projects/0219dbda54274fdc9bdccbd960c30c6b/users/02941c6ad2d54e48b1114d9e030c66c7/roles/2c7f73c184d0406bbf2899465953342b` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:17.680337 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac24fab-6856-4a1d-87e4-400b3d34b035 None admin] RBAC: Authorizing `identity:create_grant(project_id=0219dbda54274fdc9bdccbd960c30c6b, user_id=02941c6ad2d54e48b1114d9e030c66c7, role_id=2c7f73c184d0406bbf2899465953342b)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:17.682401 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac24fab-6856-4a1d-87e4-400b3d34b035 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:17.697820 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 611/1232] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jun 7 21:53:17 2022] PUT /identity/v3/projects/0219dbda54274fdc9bdccbd960c30c6b/users/02941c6ad2d54e48b1114d9e030c66c7/roles/2c7f73c184d0406bbf2899465953342b => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 21:53:19.980029 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635dcf0b-6cbc-47d1-9973-3201999708b2 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:19.980804 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635dcf0b-6cbc-47d1-9973-3201999708b2 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:19.980804 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635dcf0b-6cbc-47d1-9973-3201999708b2 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:19.982487 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 622/1233] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:53:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:53:19.989319 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb4756-91a3-45d9-b6a0-c5783b0315bc None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:19.989857 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb4756-91a3-45d9-b6a0-c5783b0315bc None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:19.990038 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb4756-91a3-45d9-b6a0-c5783b0315bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:20.011199 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-7dfb4756-91a3-45d9-b6a0-c5783b0315bc None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:53:20.013166 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-7dfb4756-91a3-45d9-b6a0-c5783b0315bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:53:20.088988 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 612/1234] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:53:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:53:20.113405 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635dcf0b-6cbc-47d1-9973-3201999708b2 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:20.139795 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:20.223737 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635dcf0b-6cbc-47d1-9973-3201999708b2 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:20.267037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de01ad8-9906-4a0c-ab73-a529ab1ecefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:20.268149 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de01ad8-9906-4a0c-ab73-a529ab1ecefb None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:20.268586 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de01ad8-9906-4a0c-ab73-a529ab1ecefb None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:20.268998 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de01ad8-9906-4a0c-ab73-a529ab1ecefb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:20.284397 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de01ad8-9906-4a0c-ab73-a529ab1ecefb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:53:20.295620 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-0de01ad8-9906-4a0c-ab73-a529ab1ecefb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 21:53:20.295620 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:53:20.295620 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:53:20.295620 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:53:20.295620 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:53:20.295620 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:53:20.295620 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:53:20.295620 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:53:20.295620 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:53:20.295620 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:53:20.295620 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:53:20.295620 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:53:20.295620 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:53:20.295620 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:53:20.295620 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:53:20.295620 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:53:20.296781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:53:20.296781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:53:20.296781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:53:20.296781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:53:20.296781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:53:20.296781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:53:20.296781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:53:20.296781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:53:20.296781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:53:20.296781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:53:20.296781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:53:20.296781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:53:20.296781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:53:20.296781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:53:20.296781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:53:20.296781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:53:20.297796 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:53:20.297796 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:53:20.297796 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:53:20.297796 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:53:20.297796 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:53:20.297796 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:53:20.297796 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:53:20.297796 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:53:20.297796 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:53:20.297796 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:53:20.297796 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 21:53:20.297796 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:53:20.297796 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 623/1235] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jun 7 21:53:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:53:20.304038 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfb4756-91a3-45d9-b6a0-c5783b0315bc None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:20.353668 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:20.412497 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfb4756-91a3-45d9-b6a0-c5783b0315bc None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:20.447338 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78de5377-3b80-4fb0-ae4b-f323e1fb8b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:20.448053 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de5377-3b80-4fb0-ae4b-f323e1fb8b95 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:20.448342 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de5377-3b80-4fb0-ae4b-f323e1fb8b95 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:20.448617 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de5377-3b80-4fb0-ae4b-f323e1fb8b95 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:20.449416 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78de5377-3b80-4fb0-ae4b-f323e1fb8b95 None admin] RBAC: Adding query filter params (name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:53:20.449794 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78de5377-3b80-4fb0-ae4b-f323e1fb8b95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:20.451229 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78de5377-3b80-4fb0-ae4b-f323e1fb8b95 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:20.460789 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 613/1236] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Jun 7 21:53:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:53:20.466767 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de01ad8-9906-4a0c-ab73-a529ab1ecefb None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:20.566227 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:20.582774 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de01ad8-9906-4a0c-ab73-a529ab1ecefb None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:20.624259 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18650654-14c6-4aee-8ab9-d7a245c18315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:20.625079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18650654-14c6-4aee-8ab9-d7a245c18315 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:20.625361 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18650654-14c6-4aee-8ab9-d7a245c18315 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:20.625636 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18650654-14c6-4aee-8ab9-d7a245c18315 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:20.639016 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18650654-14c6-4aee-8ab9-d7a245c18315 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:20.640731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18650654-14c6-4aee-8ab9-d7a245c18315 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:20.660888 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-18650654-14c6-4aee-8ab9-d7a245c18315 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:53:20.660888 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:53:20.660888 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:53:20.660888 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:53:20.660888 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:53:20.660888 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:53:20.660888 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:53:20.660888 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:53:20.660888 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:53:20.660888 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:53:20.660888 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:53:20.660888 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:53:20.660888 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:53:20.660888 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:53:20.660888 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:53:20.661924 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:53:20.661924 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:53:20.661924 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:53:20.661924 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:53:20.661924 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:53:20.661924 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:53:20.661924 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:53:20.661924 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:53:20.661924 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:53:20.661924 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:53:20.661924 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:53:20.661924 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:53:20.661924 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:53:20.661924 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:53:20.661924 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:53:20.661924 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:53:20.661924 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:53:20.663233 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:53:20.663233 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:53:20.663233 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:53:20.663233 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:53:20.663233 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 624/1237] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:53:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:53:20.668484 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78de5377-3b80-4fb0-ae4b-f323e1fb8b95 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:20.780342 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78de5377-3b80-4fb0-ae4b-f323e1fb8b95 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:20.790709 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:20.820544 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d4d860-9e57-420c-88fc-c49e6a7d80dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:20.821496 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4d860-9e57-420c-88fc-c49e6a7d80dd None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:20.822012 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4d860-9e57-420c-88fc-c49e6a7d80dd None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:20.822304 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4d860-9e57-420c-88fc-c49e6a7d80dd None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:20.823040 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d4d860-9e57-420c-88fc-c49e6a7d80dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:53:20.823373 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d4d860-9e57-420c-88fc-c49e6a7d80dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:20.824919 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d4d860-9e57-420c-88fc-c49e6a7d80dd None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:20.838581 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 614/1238] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:53:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:53:20.845952 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18650654-14c6-4aee-8ab9-d7a245c18315 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:20.956803 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18650654-14c6-4aee-8ab9-d7a245c18315 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:21.003445 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205967eb-a0f6-47c2-98d5-3957e3d59654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:21.004552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205967eb-a0f6-47c2-98d5-3957e3d59654 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:21.004891 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205967eb-a0f6-47c2-98d5-3957e3d59654 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:21.005216 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205967eb-a0f6-47c2-98d5-3957e3d59654 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:21.008728 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:21.016400 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205967eb-a0f6-47c2-98d5-3957e3d59654 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:21.018741 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205967eb-a0f6-47c2-98d5-3957e3d59654 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:21.037192 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-205967eb-a0f6-47c2-98d5-3957e3d59654 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 21:53:21.037192 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:53:21.037192 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:53:21.037192 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:53:21.037192 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:53:21.037192 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:53:21.037192 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:53:21.037192 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:53:21.037192 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:53:21.037192 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:53:21.037192 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:53:21.037192 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:53:21.037192 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 21:53:21.037192 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 21:53:21.037192 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 21:53:21.037192 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 21:53:21.038387 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:53:21.038387 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:53:21.038387 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 21:53:21.038387 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:53:21.038387 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 21:53:21.038387 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:53:21.038387 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:53:21.038387 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:53:21.038387 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:53:21.038387 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:53:21.038387 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:53:21.038387 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:53:21.038387 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:53:21.038387 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:53:21.038387 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:53:21.038387 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:53:21.038387 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:53:21.038387 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:53:21.039705 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:53:21.039705 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:53:21.039705 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 21:53:21.039705 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 21:53:21.039705 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 21:53:21.039705 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 21:53:21.039705 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 21:53:21.039705 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 21:53:21.039705 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 21:53:21.039705 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:53:21.039705 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 625/1239] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:53:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:53:21.046281 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d4d860-9e57-420c-88fc-c49e6a7d80dd None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:21.164021 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d4d860-9e57-420c-88fc-c49e6a7d80dd None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:21.206471 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700490c3-6632-403b-ba67-bc35ace8a387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:21.207156 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700490c3-6632-403b-ba67-bc35ace8a387 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:21.207476 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700490c3-6632-403b-ba67-bc35ace8a387 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:21.207755 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700490c3-6632-403b-ba67-bc35ace8a387 None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:21.208600 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700490c3-6632-403b-ba67-bc35ace8a387 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:53:21.208911 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700490c3-6632-403b-ba67-bc35ace8a387 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:21.210351 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700490c3-6632-403b-ba67-bc35ace8a387 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:21.235844 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 615/1240] 10.222.0.13 () {56 vars in 1143 bytes} [Tue Jun 7 21:53:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:53:21.237348 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205967eb-a0f6-47c2-98d5-3957e3d59654 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:21.346709 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205967eb-a0f6-47c2-98d5-3957e3d59654 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:21.381845 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ace9d2-bbd3-4aa7-b620-02540fadc4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:21.382487 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ace9d2-bbd3-4aa7-b620-02540fadc4b2 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:21.382766 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ace9d2-bbd3-4aa7-b620-02540fadc4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:21.383042 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ace9d2-bbd3-4aa7-b620-02540fadc4b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:21.446223 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:21.453324 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ace9d2-bbd3-4aa7-b620-02540fadc4b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:21.454787 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ace9d2-bbd3-4aa7-b620-02540fadc4b2 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:21.579426 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 626/1241] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:53:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 21:53:21.585248 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700490c3-6632-403b-ba67-bc35ace8a387 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:21.615907 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:21.695251 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700490c3-6632-403b-ba67-bc35ace8a387 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:21.732262 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de6e29d-c60c-4d12-9453-6b00707654e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:21.732985 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6e29d-c60c-4d12-9453-6b00707654e6 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:21.733269 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6e29d-c60c-4d12-9453-6b00707654e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:21.733548 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6e29d-c60c-4d12-9453-6b00707654e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:21.744024 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de6e29d-c60c-4d12-9453-6b00707654e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:21.745466 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de6e29d-c60c-4d12-9453-6b00707654e6 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:21.763770 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-1de6e29d-c60c-4d12-9453-6b00707654e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:53:21.763770 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:53:21.763770 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:53:21.763770 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:53:21.763770 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:53:21.763770 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:53:21.763770 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:53:21.763770 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:53:21.763770 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:53:21.763770 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:53:21.763770 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:53:21.763770 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:53:21.763770 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:53:21.763770 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:53:21.763770 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:53:21.764767 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:53:21.764767 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:53:21.764767 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:53:21.764767 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:53:21.764767 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:53:21.764767 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:53:21.764767 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:53:21.764767 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:53:21.764767 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:53:21.764767 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:53:21.764767 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:53:21.764767 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:53:21.764767 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:53:21.764767 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:53:21.764767 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:53:21.764767 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:53:21.764767 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:53:21.765681 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:53:21.765681 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:53:21.765681 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:53:21.765681 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:53:21.765681 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 616/1242] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:53:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:53:21.771022 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ace9d2-bbd3-4aa7-b620-02540fadc4b2 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:21.877361 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:21.883755 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ace9d2-bbd3-4aa7-b620-02540fadc4b2 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:21.920857 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b79c7ab-9442-42dd-8160-06666d607289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:21.921796 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79c7ab-9442-42dd-8160-06666d607289 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:21.922079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79c7ab-9442-42dd-8160-06666d607289 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:21.922357 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79c7ab-9442-42dd-8160-06666d607289 None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:21.923079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b79c7ab-9442-42dd-8160-06666d607289 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:53:21.923474 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b79c7ab-9442-42dd-8160-06666d607289 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:21.924775 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b79c7ab-9442-42dd-8160-06666d607289 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:21.938116 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 627/1243] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:53:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:53:21.943943 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de6e29d-c60c-4d12-9453-6b00707654e6 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:22.033346 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:22.059135 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de6e29d-c60c-4d12-9453-6b00707654e6 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:22.095537 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b1d674-a2fc-4d02-b6c1-0ba132811993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:22.096335 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1d674-a2fc-4d02-b6c1-0ba132811993 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:22.096621 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1d674-a2fc-4d02-b6c1-0ba132811993 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:22.098103 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1d674-a2fc-4d02-b6c1-0ba132811993 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:22.104151 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b1d674-a2fc-4d02-b6c1-0ba132811993 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:22.106297 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b1d674-a2fc-4d02-b6c1-0ba132811993 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:22.120274 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-35b1d674-a2fc-4d02-b6c1-0ba132811993 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:53:22.120274 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:53:22.120274 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:53:22.120274 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:53:22.120274 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:53:22.120274 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:53:22.120274 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:53:22.120274 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:53:22.120274 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:53:22.120274 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:53:22.120274 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:53:22.120274 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:53:22.120274 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 21:53:22.120274 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 21:53:22.120274 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 21:53:22.121523 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 21:53:22.121523 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:53:22.121523 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:53:22.121523 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:53:22.121523 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:53:22.121523 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:53:22.121523 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:53:22.121523 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:53:22.121523 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:53:22.121523 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:53:22.121523 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:53:22.121523 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:53:22.121523 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:53:22.121523 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:53:22.121523 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:53:22.121523 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:53:22.121523 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:53:22.122578 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 21:53:22.122578 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 21:53:22.122578 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 21:53:22.122578 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 21:53:22.122578 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 21:53:22.122578 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 21:53:22.122578 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:53:22.122578 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:53:22.122578 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 617/1244] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:53:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:53:22.127302 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b79c7ab-9442-42dd-8160-06666d607289 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:22.232440 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b79c7ab-9442-42dd-8160-06666d607289 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:22.266413 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bf0992-e688-465b-b7b9-4a5d283c7d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:22.267131 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf0992-e688-465b-b7b9-4a5d283c7d04 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:22.267504 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf0992-e688-465b-b7b9-4a5d283c7d04 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:22.267830 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf0992-e688-465b-b7b9-4a5d283c7d04 None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:22.268670 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bf0992-e688-465b-b7b9-4a5d283c7d04 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:53:22.269043 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bf0992-e688-465b-b7b9-4a5d283c7d04 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:22.270735 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bf0992-e688-465b-b7b9-4a5d283c7d04 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:22.284489 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 628/1245] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jun 7 21:53:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:53:22.290577 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b1d674-a2fc-4d02-b6c1-0ba132811993 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:22.420762 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b1d674-a2fc-4d02-b6c1-0ba132811993 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:22.461585 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d69465-a27c-460d-a47f-a0756a29f5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:22.462617 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d69465-a27c-460d-a47f-a0756a29f5f7 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:22.463104 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d69465-a27c-460d-a47f-a0756a29f5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:22.463627 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d69465-a27c-460d-a47f-a0756a29f5f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:22.465038 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d69465-a27c-460d-a47f-a0756a29f5f7 None admin] RBAC: Adding query filter params (role.id=2c7f73c184d0406bbf2899465953342b, scope.project.id=0219dbda54274fdc9bdccbd960c30c6b, user.id=02941c6ad2d54e48b1114d9e030c66c7) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:53:22.465586 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d69465-a27c-460d-a47f-a0756a29f5f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:22.468254 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d69465-a27c-460d-a47f-a0756a29f5f7 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:22.491061 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 618/1246] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:53:22 2022] GET /identity/v3/role_assignments?user.id=02941c6ad2d54e48b1114d9e030c66c7&scope.project.id=0219dbda54274fdc9bdccbd960c30c6b&role.id=2c7f73c184d0406bbf2899465953342b => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:53:24.888090 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d0c49-6e39-4fa1-9baf-aa14aa39c0c2 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:24.888707 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d0c49-6e39-4fa1-9baf-aa14aa39c0c2 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:24.888707 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d0c49-6e39-4fa1-9baf-aa14aa39c0c2 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:24.890149 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 629/1247] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:53:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 21:53:24.895383 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59fba00-0578-44ec-ba26-431a9564e5fe None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:24.895865 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59fba00-0578-44ec-ba26-431a9564e5fe None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:24.896030 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59fba00-0578-44ec-ba26-431a9564e5fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:24.901358 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:24.916750 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-d59fba00-0578-44ec-ba26-431a9564e5fe None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:53:24.918072 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-d59fba00-0578-44ec-ba26-431a9564e5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:53:24.994090 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 619/1248] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:53:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 21:53:25.023091 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7d0c49-6e39-4fa1-9baf-aa14aa39c0c2 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:25.045103 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:25.144468 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7d0c49-6e39-4fa1-9baf-aa14aa39c0c2 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:25.179163 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4036ea3d-0fea-4632-8216-9c8a6b6f02e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:25.179990 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036ea3d-0fea-4632-8216-9c8a6b6f02e1 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:25.180270 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036ea3d-0fea-4632-8216-9c8a6b6f02e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:25.180548 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036ea3d-0fea-4632-8216-9c8a6b6f02e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:25.191563 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4036ea3d-0fea-4632-8216-9c8a6b6f02e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:53:25.199825 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-4036ea3d-0fea-4632-8216-9c8a6b6f02e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:53:25.199825 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:53:25.199825 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:53:25.199825 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:53:25.199825 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:53:25.199825 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:53:25.199825 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:53:25.199825 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:53:25.199825 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:53:25.199825 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:53:25.199825 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:53:25.199825 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:53:25.199825 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:53:25.199825 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:53:25.199825 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:53:25.199825 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:53:25.201205 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:53:25.201205 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:53:25.201205 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:53:25.201205 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:53:25.201205 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:53:25.201205 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:53:25.201205 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:53:25.201205 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:53:25.201205 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:53:25.201205 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:53:25.201205 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:53:25.201205 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:53:25.201205 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:53:25.201205 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:53:25.201205 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:53:25.201205 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:53:25.202372 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:53:25.202372 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:53:25.202372 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:53:25.202372 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:53:25.202372 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:53:25.202372 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:53:25.202372 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:53:25.202372 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:53:25.202372 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:53:25.202372 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:53:25.202372 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:53:25.202372 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:53:25.202372 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 630/1249] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jun 7 21:53:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:53:25.207060 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59fba00-0578-44ec-ba26-431a9564e5fe None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:25.309200 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59fba00-0578-44ec-ba26-431a9564e5fe None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:25.345681 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:25.350959 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5e42d2-f52e-49ce-923a-464508e20e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:25.351668 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e42d2-f52e-49ce-923a-464508e20e1b None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:25.352006 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e42d2-f52e-49ce-923a-464508e20e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:25.352347 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e42d2-f52e-49ce-923a-464508e20e1b None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:25.353109 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5e42d2-f52e-49ce-923a-464508e20e1b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:53:25.353456 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5e42d2-f52e-49ce-923a-464508e20e1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:25.355454 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5e42d2-f52e-49ce-923a-464508e20e1b None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:25.364966 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 620/1250] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:53:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:53:25.370576 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4036ea3d-0fea-4632-8216-9c8a6b6f02e1 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:25.456595 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:25.479765 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4036ea3d-0fea-4632-8216-9c8a6b6f02e1 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:25.522362 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9936401-74f9-42ec-8f22-c3291551a4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:25.523471 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9936401-74f9-42ec-8f22-c3291551a4b2 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:25.523938 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9936401-74f9-42ec-8f22-c3291551a4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:25.524285 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9936401-74f9-42ec-8f22-c3291551a4b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:25.538113 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9936401-74f9-42ec-8f22-c3291551a4b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:25.540333 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9936401-74f9-42ec-8f22-c3291551a4b2 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:25.562686 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-a9936401-74f9-42ec-8f22-c3291551a4b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:53:25.562686 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:53:25.562686 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:53:25.562686 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:53:25.562686 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:53:25.562686 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:53:25.562686 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:53:25.562686 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:53:25.562686 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:53:25.562686 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:53:25.562686 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:53:25.562686 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:53:25.562686 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:53:25.562686 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:53:25.562686 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:53:25.563789 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:53:25.563789 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:53:25.563789 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:53:25.563789 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:53:25.563789 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:53:25.563789 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:53:25.563789 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:53:25.563789 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:53:25.563789 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:53:25.563789 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:53:25.563789 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:53:25.563789 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:53:25.563789 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:53:25.563789 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:53:25.563789 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:53:25.563789 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:53:25.563789 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:53:25.564709 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:53:25.564709 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:53:25.564709 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:53:25.564709 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:53:25.564709 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 631/1251] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:53:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:53:25.570060 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5e42d2-f52e-49ce-923a-464508e20e1b None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:25.679591 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5e42d2-f52e-49ce-923a-464508e20e1b None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:25.714286 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34fb260-c922-41d6-af86-77e83d2edaea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:25.715160 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34fb260-c922-41d6-af86-77e83d2edaea None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:25.715481 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34fb260-c922-41d6-af86-77e83d2edaea None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:25.715760 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34fb260-c922-41d6-af86-77e83d2edaea None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:25.716477 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34fb260-c922-41d6-af86-77e83d2edaea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:53:25.716783 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34fb260-c922-41d6-af86-77e83d2edaea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:25.718139 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34fb260-c922-41d6-af86-77e83d2edaea None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:25.730667 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 621/1252] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:53:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:53:25.736797 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9936401-74f9-42ec-8f22-c3291551a4b2 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:25.849332 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9936401-74f9-42ec-8f22-c3291551a4b2 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:25.876962 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:25.887800 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d92c42-06d4-470f-b82d-ea7f558a70bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:25.888609 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d92c42-06d4-470f-b82d-ea7f558a70bd None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:25.889191 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d92c42-06d4-470f-b82d-ea7f558a70bd None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:25.889191 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d92c42-06d4-470f-b82d-ea7f558a70bd None admin] PATH_INFO: `/v3/users/placement` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:25.897641 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d92c42-06d4-470f-b82d-ea7f558a70bd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:25.899346 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d92c42-06d4-470f-b82d-ea7f558a70bd None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:25.915662 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-24d92c42-06d4-470f-b82d-ea7f558a70bd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 21:53:25.915662 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:53:25.915662 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:53:25.915662 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:53:25.915662 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:53:25.915662 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:53:25.915662 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:53:25.915662 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:53:25.915662 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:53:25.915662 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:53:25.915662 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:53:25.915662 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:53:25.915662 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 21:53:25.915662 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 21:53:25.915662 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 21:53:25.915662 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 21:53:25.916866 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:53:25.916866 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:53:25.916866 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 21:53:25.916866 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:53:25.916866 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 21:53:25.916866 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:53:25.916866 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:53:25.916866 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:53:25.916866 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:53:25.916866 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:53:25.916866 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:53:25.916866 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:53:25.916866 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:53:25.916866 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:53:25.916866 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:53:25.916866 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:53:25.916866 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:53:25.916866 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:53:25.918263 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:53:25.918263 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:53:25.918263 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 21:53:25.918263 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 21:53:25.918263 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 21:53:25.918263 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 21:53:25.918263 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 21:53:25.918263 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 21:53:25.918263 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 21:53:25.918263 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:53:25.918263 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 632/1253] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:53:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:53:25.922874 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34fb260-c922-41d6-af86-77e83d2edaea None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:25.982550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:26.031887 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34fb260-c922-41d6-af86-77e83d2edaea None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:26.072527 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca314104-a69d-4f30-a312-f2ffaf07e3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:26.073200 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca314104-a69d-4f30-a312-f2ffaf07e3cb None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:26.073484 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca314104-a69d-4f30-a312-f2ffaf07e3cb None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:26.073817 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca314104-a69d-4f30-a312-f2ffaf07e3cb None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:26.075567 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca314104-a69d-4f30-a312-f2ffaf07e3cb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:53:26.075876 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca314104-a69d-4f30-a312-f2ffaf07e3cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:26.077612 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca314104-a69d-4f30-a312-f2ffaf07e3cb None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:26.101496 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 622/1254] 10.222.0.13 () {56 vars in 1143 bytes} [Tue Jun 7 21:53:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:53:26.102629 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d92c42-06d4-470f-b82d-ea7f558a70bd None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:26.214104 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d92c42-06d4-470f-b82d-ea7f558a70bd None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:26.251064 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4313bd-30d7-4689-8ee4-2c5b3df43e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:26.251747 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4313bd-30d7-4689-8ee4-2c5b3df43e02 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:26.252032 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4313bd-30d7-4689-8ee4-2c5b3df43e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:26.252309 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4313bd-30d7-4689-8ee4-2c5b3df43e02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:26.316002 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:26.330494 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4313bd-30d7-4689-8ee4-2c5b3df43e02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:26.332325 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4313bd-30d7-4689-8ee4-2c5b3df43e02 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:26.441329 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 633/1255] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:53:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 21:53:26.446513 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca314104-a69d-4f30-a312-f2ffaf07e3cb None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:26.571716 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca314104-a69d-4f30-a312-f2ffaf07e3cb None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:26.587654 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:26.613364 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb449dac-a92e-4b52-a937-070e3124689c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:26.614163 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb449dac-a92e-4b52-a937-070e3124689c None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:26.614445 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb449dac-a92e-4b52-a937-070e3124689c None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:26.614713 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb449dac-a92e-4b52-a937-070e3124689c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:26.625989 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb449dac-a92e-4b52-a937-070e3124689c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:26.627491 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb449dac-a92e-4b52-a937-070e3124689c None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:26.645981 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-fb449dac-a92e-4b52-a937-070e3124689c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:53:26.645981 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:53:26.645981 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:53:26.645981 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:53:26.645981 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:53:26.645981 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:53:26.645981 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:53:26.645981 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:53:26.645981 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:53:26.645981 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:53:26.645981 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:53:26.645981 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:53:26.645981 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:53:26.645981 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:53:26.645981 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:53:26.647161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:53:26.647161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:53:26.647161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:53:26.647161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:53:26.647161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:53:26.647161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:53:26.647161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:53:26.647161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:53:26.647161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:53:26.647161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:53:26.647161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:53:26.647161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:53:26.647161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:53:26.647161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:53:26.647161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:53:26.647161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:53:26.647161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:53:26.648287 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:53:26.648287 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:53:26.648287 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:53:26.648287 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:53:26.648287 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 623/1256] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:53:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:53:26.653834 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4313bd-30d7-4689-8ee4-2c5b3df43e02 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:26.746593 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:26.760751 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4313bd-30d7-4689-8ee4-2c5b3df43e02 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:26.797047 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151cfea-0e4e-464a-bfc8-5fbd21a9a4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:26.797726 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151cfea-0e4e-464a-bfc8-5fbd21a9a4e4 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:26.798005 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151cfea-0e4e-464a-bfc8-5fbd21a9a4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:26.798277 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151cfea-0e4e-464a-bfc8-5fbd21a9a4e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:26.799006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d151cfea-0e4e-464a-bfc8-5fbd21a9a4e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:53:26.799310 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d151cfea-0e4e-464a-bfc8-5fbd21a9a4e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:26.800718 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d151cfea-0e4e-464a-bfc8-5fbd21a9a4e4 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:26.813537 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 634/1257] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:53:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:53:26.819424 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb449dac-a92e-4b52-a937-070e3124689c None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:26.924737 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb449dac-a92e-4b52-a937-070e3124689c None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:26.961322 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed98982a-590f-4801-8bd6-47ebb022b7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:26.962119 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed98982a-590f-4801-8bd6-47ebb022b7ed None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:26.962402 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed98982a-590f-4801-8bd6-47ebb022b7ed None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:26.962676 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed98982a-590f-4801-8bd6-47ebb022b7ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:26.969731 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed98982a-590f-4801-8bd6-47ebb022b7ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:26.971167 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed98982a-590f-4801-8bd6-47ebb022b7ed None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:26.985540 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-ed98982a-590f-4801-8bd6-47ebb022b7ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:53:26.985540 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:53:26.985540 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:53:26.985540 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:53:26.985540 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:53:26.985540 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:53:26.985540 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:53:26.985540 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:53:26.985540 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:53:26.985540 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:53:26.985540 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:53:26.985540 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:53:26.985540 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 21:53:26.985540 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 21:53:26.985540 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 21:53:26.986625 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 21:53:26.986625 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:53:26.986625 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:53:26.986625 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:53:26.986625 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:53:26.986625 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:53:26.986625 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:53:26.986625 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:53:26.986625 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:53:26.986625 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:53:26.986625 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:53:26.986625 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:53:26.986625 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:53:26.986625 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:53:26.986625 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:53:26.986625 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:53:26.986625 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:53:26.987941 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 21:53:26.987941 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 21:53:26.987941 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 21:53:26.987941 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 21:53:26.987941 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 21:53:26.987941 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 21:53:26.987941 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:53:26.987941 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:53:26.987941 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 624/1258] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:53:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 21:53:26.994392 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151cfea-0e4e-464a-bfc8-5fbd21a9a4e4 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:27.097595 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151cfea-0e4e-464a-bfc8-5fbd21a9a4e4 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:27.130802 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e81245f-2469-4045-a49d-994b0df33130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:27.131492 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e81245f-2469-4045-a49d-994b0df33130 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:27.131775 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e81245f-2469-4045-a49d-994b0df33130 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:27.132050 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e81245f-2469-4045-a49d-994b0df33130 None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:27.132835 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e81245f-2469-4045-a49d-994b0df33130 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:53:27.133145 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e81245f-2469-4045-a49d-994b0df33130 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:27.134565 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e81245f-2469-4045-a49d-994b0df33130 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:27.147552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 635/1259] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jun 7 21:53:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:53:27.154339 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed98982a-590f-4801-8bd6-47ebb022b7ed None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:27.180766 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:27.259217 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed98982a-590f-4801-8bd6-47ebb022b7ed None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:27.292024 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194a7081-6e7b-48eb-a15f-22b6d0676674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:27.293260 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a7081-6e7b-48eb-a15f-22b6d0676674 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:27.293541 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a7081-6e7b-48eb-a15f-22b6d0676674 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:27.293818 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a7081-6e7b-48eb-a15f-22b6d0676674 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:27.294644 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194a7081-6e7b-48eb-a15f-22b6d0676674 None admin] RBAC: Adding query filter params (role.id=6b483758dbd44eb4a78df2ce4603d601, scope.project.id=0219dbda54274fdc9bdccbd960c30c6b, user.id=02941c6ad2d54e48b1114d9e030c66c7) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:53:27.294955 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194a7081-6e7b-48eb-a15f-22b6d0676674 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:27.296409 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194a7081-6e7b-48eb-a15f-22b6d0676674 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:27.301712 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 625/1260] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:53:27 2022] GET /identity/v3/role_assignments?user.id=02941c6ad2d54e48b1114d9e030c66c7&scope.project.id=0219dbda54274fdc9bdccbd960c30c6b&role.id=6b483758dbd44eb4a78df2ce4603d601 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:53:29.567692 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a13d6-7552-4951-a597-56aa1d0e0968 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:29.568251 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a13d6-7552-4951-a597-56aa1d0e0968 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:29.568251 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a13d6-7552-4951-a597-56aa1d0e0968 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:29.569607 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 636/1261] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:53:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:53:29.574972 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b4d4ba-ff25-44c3-b1b0-b38f51e2b290 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:29.575377 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b4d4ba-ff25-44c3-b1b0-b38f51e2b290 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:29.575556 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b4d4ba-ff25-44c3-b1b0-b38f51e2b290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:29.592493 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-b8b4d4ba-ff25-44c3-b1b0-b38f51e2b290 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:53:29.593685 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-b8b4d4ba-ff25-44c3-b1b0-b38f51e2b290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:53:29.658332 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 626/1262] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:53:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:53:29.681368 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9a13d6-7552-4951-a597-56aa1d0e0968 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:29.692136 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:29.781215 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9a13d6-7552-4951-a597-56aa1d0e0968 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:29.813872 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0306bf52-4f6e-427f-ba5d-48b467c1a823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:29.814653 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0306bf52-4f6e-427f-ba5d-48b467c1a823 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:29.814937 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0306bf52-4f6e-427f-ba5d-48b467c1a823 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:29.815213 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0306bf52-4f6e-427f-ba5d-48b467c1a823 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:29.825240 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0306bf52-4f6e-427f-ba5d-48b467c1a823 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:53:29.832794 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-0306bf52-4f6e-427f-ba5d-48b467c1a823 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:53:29.832794 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:53:29.832794 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:53:29.832794 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:53:29.832794 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:53:29.832794 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:53:29.832794 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:53:29.832794 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:53:29.832794 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:53:29.832794 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:53:29.832794 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:53:29.832794 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:53:29.832794 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:53:29.832794 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:53:29.832794 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:53:29.832794 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:53:29.833878 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:53:29.833878 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:53:29.833878 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:53:29.833878 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:53:29.833878 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:53:29.833878 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:53:29.833878 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:53:29.833878 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:53:29.833878 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:53:29.833878 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:53:29.833878 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:53:29.833878 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:53:29.833878 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:53:29.833878 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:53:29.833878 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:53:29.833878 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:53:29.834768 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:53:29.834768 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:53:29.834768 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:53:29.834768 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:53:29.834768 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:53:29.834768 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:53:29.834768 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:53:29.834768 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:53:29.834768 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:53:29.834768 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:53:29.834768 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:53:29.834768 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:53:29.834768 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 637/1263] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jun 7 21:53:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:53:29.840587 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b4d4ba-ff25-44c3-b1b0-b38f51e2b290 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:29.859800 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:29.937246 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b4d4ba-ff25-44c3-b1b0-b38f51e2b290 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:29.970245 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d837e-b12c-4b96-b298-19d2770c14bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:29.970899 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d837e-b12c-4b96-b298-19d2770c14bc None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:29.971181 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d837e-b12c-4b96-b298-19d2770c14bc None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:29.971486 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d837e-b12c-4b96-b298-19d2770c14bc None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:29.972230 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d837e-b12c-4b96-b298-19d2770c14bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:53:29.972787 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d837e-b12c-4b96-b298-19d2770c14bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:29.974154 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d837e-b12c-4b96-b298-19d2770c14bc None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:29.982100 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 627/1264] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:53:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:53:29.987826 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0306bf52-4f6e-427f-ba5d-48b467c1a823 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:30.051900 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:30.086689 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0306bf52-4f6e-427f-ba5d-48b467c1a823 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:30.119363 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b185bf6-d06f-4dad-9583-92aa9a6cbf59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:30.120139 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b185bf6-d06f-4dad-9583-92aa9a6cbf59 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:30.120443 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b185bf6-d06f-4dad-9583-92aa9a6cbf59 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:30.120818 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b185bf6-d06f-4dad-9583-92aa9a6cbf59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:30.130921 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b185bf6-d06f-4dad-9583-92aa9a6cbf59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:30.132381 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b185bf6-d06f-4dad-9583-92aa9a6cbf59 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:30.149234 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-8b185bf6-d06f-4dad-9583-92aa9a6cbf59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:53:30.149234 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:53:30.149234 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:53:30.149234 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:53:30.149234 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:53:30.149234 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:53:30.149234 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:53:30.149234 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:53:30.149234 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:53:30.149234 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:53:30.149234 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:53:30.149234 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:53:30.149234 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:53:30.149234 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:53:30.149234 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:53:30.150276 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:53:30.150276 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:53:30.150276 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:53:30.150276 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:53:30.150276 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:53:30.150276 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:53:30.150276 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:53:30.150276 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:53:30.150276 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:53:30.150276 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:53:30.150276 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:53:30.150276 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:53:30.150276 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:53:30.150276 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:53:30.150276 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:53:30.150276 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:53:30.150276 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:53:30.151243 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:53:30.151243 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:53:30.151243 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:53:30.151243 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:53:30.151243 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 638/1265] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:53:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:53:30.156327 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d837e-b12c-4b96-b298-19d2770c14bc None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:30.235712 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:30.255263 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d837e-b12c-4b96-b298-19d2770c14bc None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:30.287996 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ae712-0d38-42c7-ab25-ed9110cb13ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:30.288652 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ae712-0d38-42c7-ab25-ed9110cb13ad None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:30.288939 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ae712-0d38-42c7-ab25-ed9110cb13ad None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:30.289220 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ae712-0d38-42c7-ab25-ed9110cb13ad None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:30.289938 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ae712-0d38-42c7-ab25-ed9110cb13ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:53:30.290246 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ae712-0d38-42c7-ab25-ed9110cb13ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:30.291674 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ae712-0d38-42c7-ab25-ed9110cb13ad None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:30.304284 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 628/1266] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:53:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:53:30.310000 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b185bf6-d06f-4dad-9583-92aa9a6cbf59 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:30.405247 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b185bf6-d06f-4dad-9583-92aa9a6cbf59 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:30.424412 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:30.438342 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cb78e8-31e7-4cd7-a43f-d06ec4d56ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:30.439129 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb78e8-31e7-4cd7-a43f-d06ec4d56ad6 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:30.439445 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb78e8-31e7-4cd7-a43f-d06ec4d56ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:30.439718 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb78e8-31e7-4cd7-a43f-d06ec4d56ad6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:30.447639 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cb78e8-31e7-4cd7-a43f-d06ec4d56ad6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:30.449331 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cb78e8-31e7-4cd7-a43f-d06ec4d56ad6 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:30.463024 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-e9cb78e8-31e7-4cd7-a43f-d06ec4d56ad6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 21:53:30.463024 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:53:30.463024 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:53:30.463024 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:53:30.463024 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:53:30.463024 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:53:30.463024 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:53:30.463024 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:53:30.463024 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:53:30.463024 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:53:30.463024 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:53:30.463024 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:53:30.463024 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 21:53:30.463024 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 21:53:30.463024 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 21:53:30.463024 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 21:53:30.464682 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:53:30.464682 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:53:30.464682 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 21:53:30.464682 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:53:30.464682 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 21:53:30.464682 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:53:30.464682 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:53:30.464682 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:53:30.464682 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:53:30.464682 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:53:30.464682 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:53:30.464682 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:53:30.464682 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:53:30.464682 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:53:30.464682 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:53:30.464682 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:53:30.464682 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:53:30.464682 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:53:30.465893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:53:30.465893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:53:30.465893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 21:53:30.465893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 21:53:30.465893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 21:53:30.465893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 21:53:30.465893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 21:53:30.465893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 21:53:30.465893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 21:53:30.465893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:53:30.465893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 639/1267] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:53:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:53:30.470350 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ae712-0d38-42c7-ab25-ed9110cb13ad None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:30.588738 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ae712-0d38-42c7-ab25-ed9110cb13ad None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:30.622862 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7096073b-3841-43fb-a90d-ed6feb57d908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:30.623552 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7096073b-3841-43fb-a90d-ed6feb57d908 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:30.623840 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7096073b-3841-43fb-a90d-ed6feb57d908 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:30.624118 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7096073b-3841-43fb-a90d-ed6feb57d908 None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:30.624947 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7096073b-3841-43fb-a90d-ed6feb57d908 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:53:30.625255 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7096073b-3841-43fb-a90d-ed6feb57d908 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:30.626680 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7096073b-3841-43fb-a90d-ed6feb57d908 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:30.634977 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:30.644642 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 629/1268] 10.222.0.13 () {56 vars in 1143 bytes} [Tue Jun 7 21:53:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:53:30.650202 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cb78e8-31e7-4cd7-a43f-d06ec4d56ad6 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:30.747179 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cb78e8-31e7-4cd7-a43f-d06ec4d56ad6 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:30.780925 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64091b64-ec4b-4b4b-9ca6-d9866ac46a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:30.781708 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64091b64-ec4b-4b4b-9ca6-d9866ac46a2c None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:30.781984 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64091b64-ec4b-4b4b-9ca6-d9866ac46a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:30.782252 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64091b64-ec4b-4b4b-9ca6-d9866ac46a2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:30.792592 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64091b64-ec4b-4b4b-9ca6-d9866ac46a2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:30.794138 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64091b64-ec4b-4b4b-9ca6-d9866ac46a2c None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:30.810773 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-64091b64-ec4b-4b4b-9ca6-d9866ac46a2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:53:30.810773 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:53:30.810773 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:53:30.810773 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:53:30.810773 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:53:30.810773 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:53:30.810773 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:53:30.810773 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:53:30.810773 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:53:30.810773 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:53:30.810773 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:53:30.810773 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:53:30.810773 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:53:30.810773 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:53:30.810773 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:53:30.811735 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:53:30.811735 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:53:30.811735 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:53:30.811735 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:53:30.811735 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:53:30.811735 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:53:30.811735 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:53:30.811735 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:53:30.811735 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:53:30.811735 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:53:30.811735 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:53:30.811735 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:53:30.811735 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:53:30.811735 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:53:30.811735 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:53:30.811735 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:53:30.811735 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:53:30.812677 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:53:30.812677 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:53:30.812677 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:53:30.812677 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:53:30.812677 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 640/1269] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:53:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:53:30.817552 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7096073b-3841-43fb-a90d-ed6feb57d908 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:30.915027 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7096073b-3841-43fb-a90d-ed6feb57d908 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:30.947167 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59fee7b-9ce1-485f-9864-3acb03edf416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:30.947841 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59fee7b-9ce1-485f-9864-3acb03edf416 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:30.948122 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59fee7b-9ce1-485f-9864-3acb03edf416 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:30.948396 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59fee7b-9ce1-485f-9864-3acb03edf416 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:30.949703 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59fee7b-9ce1-485f-9864-3acb03edf416 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:53:30.950018 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59fee7b-9ce1-485f-9864-3acb03edf416 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:30.951389 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59fee7b-9ce1-485f-9864-3acb03edf416 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:30.962133 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 630/1270] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:53:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:53:30.968350 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64091b64-ec4b-4b4b-9ca6-d9866ac46a2c None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:30.999887 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:31.069142 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64091b64-ec4b-4b4b-9ca6-d9866ac46a2c None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:31.106729 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23266f38-6986-4ceb-baaf-3745fa43e246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:31.107533 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23266f38-6986-4ceb-baaf-3745fa43e246 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:31.107811 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23266f38-6986-4ceb-baaf-3745fa43e246 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:31.108078 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23266f38-6986-4ceb-baaf-3745fa43e246 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:31.114548 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23266f38-6986-4ceb-baaf-3745fa43e246 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:31.116014 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23266f38-6986-4ceb-baaf-3745fa43e246 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:31.130737 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-23266f38-6986-4ceb-baaf-3745fa43e246 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:53:31.130737 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:53:31.130737 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:53:31.130737 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:53:31.130737 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:53:31.130737 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:53:31.130737 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:53:31.130737 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:53:31.130737 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:53:31.130737 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:53:31.130737 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:53:31.130737 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:53:31.130737 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 21:53:31.130737 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 21:53:31.130737 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 21:53:31.131847 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 21:53:31.131847 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:53:31.131847 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:53:31.131847 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:53:31.131847 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:53:31.131847 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:53:31.131847 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:53:31.131847 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:53:31.131847 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:53:31.131847 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:53:31.131847 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:53:31.131847 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:53:31.131847 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:53:31.131847 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:53:31.131847 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:53:31.131847 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:53:31.131847 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:53:31.133169 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 21:53:31.133169 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 21:53:31.133169 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 21:53:31.133169 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 21:53:31.133169 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 21:53:31.133169 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 21:53:31.133169 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:53:31.133169 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:53:31.133169 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 641/1271] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:53:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:53:31.138763 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59fee7b-9ce1-485f-9864-3acb03edf416 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:31.194771 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:31.236574 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59fee7b-9ce1-485f-9864-3acb03edf416 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:31.269996 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52d95c9-f773-4c55-be17-bafc115fcbbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:31.270590 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52d95c9-f773-4c55-be17-bafc115fcbbb None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:31.270874 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52d95c9-f773-4c55-be17-bafc115fcbbb None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:31.271151 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52d95c9-f773-4c55-be17-bafc115fcbbb None admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:31.271961 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52d95c9-f773-4c55-be17-bafc115fcbbb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:53:31.272270 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52d95c9-f773-4c55-be17-bafc115fcbbb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:31.273687 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52d95c9-f773-4c55-be17-bafc115fcbbb None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:31.285580 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 631/1272] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jun 7 21:53:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:53:31.291493 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23266f38-6986-4ceb-baaf-3745fa43e246 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:31.380404 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:31.389289 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23266f38-6986-4ceb-baaf-3745fa43e246 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:31.434293 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3103d4-8f77-426a-82f7-c51af796ed75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:31.435182 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3103d4-8f77-426a-82f7-c51af796ed75 None admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:31.435495 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3103d4-8f77-426a-82f7-c51af796ed75 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:31.435773 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3103d4-8f77-426a-82f7-c51af796ed75 None admin] PATH_INFO: `/v3/projects/0219dbda54274fdc9bdccbd960c30c6b/users/02941c6ad2d54e48b1114d9e030c66c7/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:31.437725 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3103d4-8f77-426a-82f7-c51af796ed75 None admin] RBAC: Authorizing `identity:create_grant(project_id=0219dbda54274fdc9bdccbd960c30c6b, user_id=02941c6ad2d54e48b1114d9e030c66c7, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:31.439152 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3103d4-8f77-426a-82f7-c51af796ed75 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:31.451971 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 642/1273] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jun 7 21:53:31 2022] PUT /identity/v3/projects/0219dbda54274fdc9bdccbd960c30c6b/users/02941c6ad2d54e48b1114d9e030c66c7/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 21:53:33.728834 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba2ad7-2c38-4f0c-99ca-8cd60f579a7e None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:33.729389 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba2ad7-2c38-4f0c-99ca-8cd60f579a7e None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:33.729389 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba2ad7-2c38-4f0c-99ca-8cd60f579a7e None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:33.731182 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 632/1274] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:53:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:53:33.736347 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3dca2-d1af-426c-a2a8-e1382fe80584 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:33.736779 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3dca2-d1af-426c-a2a8-e1382fe80584 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:33.736904 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3dca2-d1af-426c-a2a8-e1382fe80584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:33.753317 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-edb3dca2-d1af-426c-a2a8-e1382fe80584 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:53:33.754504 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-edb3dca2-d1af-426c-a2a8-e1382fe80584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:53:33.820809 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 643/1275] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:53:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:53:33.845652 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ba2ad7-2c38-4f0c-99ca-8cd60f579a7e None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:33.944824 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ba2ad7-2c38-4f0c-99ca-8cd60f579a7e None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:33.962153 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:33.979469 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405389d5-16ce-4ab4-90a9-693e9de90d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:33.980481 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405389d5-16ce-4ab4-90a9-693e9de90d71 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:33.980769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405389d5-16ce-4ab4-90a9-693e9de90d71 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:33.981043 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405389d5-16ce-4ab4-90a9-693e9de90d71 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:33.990714 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405389d5-16ce-4ab4-90a9-693e9de90d71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 21:53:33.999094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-405389d5-16ce-4ab4-90a9-693e9de90d71 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:53:33.999094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:53:33.999094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:53:33.999094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:53:33.999094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:53:33.999094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:53:33.999094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:53:33.999094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:53:33.999094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:53:33.999094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:53:33.999094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:53:33.999094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:53:33.999094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 21:53:33.999094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 21:53:33.999094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 21:53:33.999094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 21:53:34.000292 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 21:53:34.000292 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 21:53:34.000292 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 21:53:34.000292 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 21:53:34.000292 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:53:34.000292 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:53:34.000292 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:53:34.000292 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:53:34.000292 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:53:34.000292 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:53:34.000292 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:53:34.000292 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:53:34.000292 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:53:34.000292 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:53:34.000292 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:53:34.000292 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:53:34.001531 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:53:34.001531 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:53:34.001531 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:53:34.001531 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:53:34.001531 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 21:53:34.001531 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 21:53:34.001531 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 21:53:34.001531 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 21:53:34.001531 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 21:53:34.001531 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 21:53:34.001531 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 21:53:34.001531 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:53:34.001531 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 633/1276] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jun 7 21:53:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:53:34.006231 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb3dca2-d1af-426c-a2a8-e1382fe80584 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:34.084957 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:34.103594 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb3dca2-d1af-426c-a2a8-e1382fe80584 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:34.137485 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9c0eab-c708-4d79-953a-a37c170188b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:34.138146 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c0eab-c708-4d79-953a-a37c170188b7 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:34.138424 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c0eab-c708-4d79-953a-a37c170188b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:34.138694 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c0eab-c708-4d79-953a-a37c170188b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:34.139556 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9c0eab-c708-4d79-953a-a37c170188b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:53:34.139862 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9c0eab-c708-4d79-953a-a37c170188b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:34.141238 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9c0eab-c708-4d79-953a-a37c170188b7 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:34.149072 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 644/1277] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jun 7 21:53:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:53:34.155291 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405389d5-16ce-4ab4-90a9-693e9de90d71 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:34.253026 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405389d5-16ce-4ab4-90a9-693e9de90d71 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:34.285425 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb3029-cbe4-4c52-b612-3b6a6dac17ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:34.286192 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb3029-cbe4-4c52-b612-3b6a6dac17ce None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:34.286472 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb3029-cbe4-4c52-b612-3b6a6dac17ce None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:34.286745 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb3029-cbe4-4c52-b612-3b6a6dac17ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:34.297215 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb3029-cbe4-4c52-b612-3b6a6dac17ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:34.298668 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb3029-cbe4-4c52-b612-3b6a6dac17ce None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:34.315547 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-49fb3029-cbe4-4c52-b612-3b6a6dac17ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:53:34.315547 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:53:34.315547 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:53:34.315547 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:53:34.315547 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:53:34.315547 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:53:34.315547 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:53:34.315547 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:53:34.315547 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:53:34.315547 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:53:34.315547 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:53:34.315547 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:53:34.315547 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:53:34.315547 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:53:34.315547 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:53:34.316577 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:53:34.316577 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:53:34.316577 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:53:34.316577 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:53:34.316577 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:53:34.316577 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:53:34.316577 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:53:34.316577 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:53:34.316577 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:53:34.316577 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:53:34.316577 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:53:34.316577 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:53:34.316577 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:53:34.316577 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:53:34.316577 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:53:34.316577 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:53:34.316577 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:53:34.317644 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:53:34.317644 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:53:34.317644 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:53:34.317644 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:53:34.317644 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 634/1278] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:53:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:53:34.322296 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9c0eab-c708-4d79-953a-a37c170188b7 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:34.564911 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9c0eab-c708-4d79-953a-a37c170188b7 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:34.597526 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba1774-3ea7-4626-821b-09c06a65c81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:34.598194 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba1774-3ea7-4626-821b-09c06a65c81a None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:34.598475 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba1774-3ea7-4626-821b-09c06a65c81a None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:34.598747 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba1774-3ea7-4626-821b-09c06a65c81a None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:34.599498 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba1774-3ea7-4626-821b-09c06a65c81a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:53:34.599800 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba1774-3ea7-4626-821b-09c06a65c81a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:34.601159 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba1774-3ea7-4626-821b-09c06a65c81a None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:34.612723 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 645/1279] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:53:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:53:34.618322 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb3029-cbe4-4c52-b612-3b6a6dac17ce None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:34.648880 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:34.717183 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb3029-cbe4-4c52-b612-3b6a6dac17ce None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:34.750067 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c9032-0f6a-4468-88bd-8594bebf63ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:34.750848 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c9032-0f6a-4468-88bd-8594bebf63ed None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:34.751126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c9032-0f6a-4468-88bd-8594bebf63ed None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:34.751425 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c9032-0f6a-4468-88bd-8594bebf63ed None admin] PATH_INFO: `/v3/users/placement` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:34.758926 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c9032-0f6a-4468-88bd-8594bebf63ed None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:34.760400 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c9032-0f6a-4468-88bd-8594bebf63ed None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:34.774296 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-c41c9032-0f6a-4468-88bd-8594bebf63ed None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 21:53:34.774296 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:53:34.774296 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:53:34.774296 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:53:34.774296 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:53:34.774296 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:53:34.774296 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:53:34.774296 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:53:34.774296 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:53:34.774296 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:53:34.774296 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:53:34.774296 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:53:34.774296 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 21:53:34.774296 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 21:53:34.774296 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 21:53:34.774296 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 21:53:34.775560 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:53:34.775560 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:53:34.775560 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 21:53:34.775560 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:53:34.775560 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 21:53:34.775560 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 21:53:34.775560 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:53:34.775560 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:53:34.775560 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:53:34.775560 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:53:34.775560 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:53:34.775560 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:53:34.775560 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:53:34.775560 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:53:34.775560 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:53:34.775560 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:53:34.775560 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:53:34.775560 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:53:34.777190 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:53:34.777190 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:53:34.777190 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 21:53:34.777190 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 21:53:34.777190 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 21:53:34.777190 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 21:53:34.777190 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 21:53:34.777190 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 21:53:34.777190 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 21:53:34.777190 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:53:34.777190 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 635/1280] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:53:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:53:34.781753 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba1774-3ea7-4626-821b-09c06a65c81a None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:34.785538 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:34.879569 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba1774-3ea7-4626-821b-09c06a65c81a None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:34.912588 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cbb90a-d2a5-423b-9ae0-29bb63c290ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:34.913235 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbb90a-d2a5-423b-9ae0-29bb63c290ab None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:34.913512 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbb90a-d2a5-423b-9ae0-29bb63c290ab None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:34.913780 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbb90a-d2a5-423b-9ae0-29bb63c290ab None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:34.914595 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cbb90a-d2a5-423b-9ae0-29bb63c290ab None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:53:34.914894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cbb90a-d2a5-423b-9ae0-29bb63c290ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:34.916410 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cbb90a-d2a5-423b-9ae0-29bb63c290ab None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:34.934184 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 646/1281] 10.222.0.13 () {56 vars in 1143 bytes} [Tue Jun 7 21:53:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:53:34.940052 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c9032-0f6a-4468-88bd-8594bebf63ed None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:35.028935 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:35.038815 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c9032-0f6a-4468-88bd-8594bebf63ed None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:35.071418 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f878f2a4-06c0-4672-af49-a305ac90b4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:35.072063 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f878f2a4-06c0-4672-af49-a305ac90b4ec None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:35.072351 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f878f2a4-06c0-4672-af49-a305ac90b4ec None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:35.072632 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f878f2a4-06c0-4672-af49-a305ac90b4ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:35.141017 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f878f2a4-06c0-4672-af49-a305ac90b4ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:35.142395 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f878f2a4-06c0-4672-af49-a305ac90b4ec None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:35.227587 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:35.238971 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 636/1282] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jun 7 21:53:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 21:53:35.244917 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cbb90a-d2a5-423b-9ae0-29bb63c290ab None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:35.303615 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:35.340959 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cbb90a-d2a5-423b-9ae0-29bb63c290ab None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:35.373869 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dec55ff-08d4-4541-b87c-b338d7a27828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:35.374642 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec55ff-08d4-4541-b87c-b338d7a27828 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:35.374922 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec55ff-08d4-4541-b87c-b338d7a27828 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:35.375194 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec55ff-08d4-4541-b87c-b338d7a27828 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:35.384872 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dec55ff-08d4-4541-b87c-b338d7a27828 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:35.386338 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dec55ff-08d4-4541-b87c-b338d7a27828 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:35.403051 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-8dec55ff-08d4-4541-b87c-b338d7a27828 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:53:35.403051 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:53:35.403051 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:53:35.403051 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:53:35.403051 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:53:35.403051 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:53:35.403051 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:53:35.403051 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:53:35.403051 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:53:35.403051 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:53:35.403051 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:53:35.403051 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:53:35.403051 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 21:53:35.403051 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 21:53:35.403051 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 21:53:35.404090 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 21:53:35.404090 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:53:35.404090 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:53:35.404090 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:53:35.404090 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:53:35.404090 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:53:35.404090 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:53:35.404090 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:53:35.404090 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:53:35.404090 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:53:35.404090 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:53:35.404090 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:53:35.404090 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:53:35.404090 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:53:35.404090 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:53:35.404090 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:53:35.404090 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:53:35.405195 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 21:53:35.405195 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 21:53:35.405195 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 21:53:35.405195 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:53:35.405195 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 647/1283] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:53:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:53:35.409797 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f878f2a4-06c0-4672-af49-a305ac90b4ec None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:35.508112 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f878f2a4-06c0-4672-af49-a305ac90b4ec None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:35.541525 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfbd1e4-b877-4cc4-941b-90c8c04293d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:35.542713 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfbd1e4-b877-4cc4-941b-90c8c04293d1 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:35.543227 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfbd1e4-b877-4cc4-941b-90c8c04293d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:35.543647 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfbd1e4-b877-4cc4-941b-90c8c04293d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:35.544364 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfbd1e4-b877-4cc4-941b-90c8c04293d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:53:35.544670 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfbd1e4-b877-4cc4-941b-90c8c04293d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:35.546024 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfbd1e4-b877-4cc4-941b-90c8c04293d1 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:35.556966 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 637/1284] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jun 7 21:53:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:53:35.562665 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dec55ff-08d4-4541-b87c-b338d7a27828 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:35.661981 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dec55ff-08d4-4541-b87c-b338d7a27828 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:35.675463 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:35.695742 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b649a3-7157-474e-81fb-2023ddff7963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:35.696529 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b649a3-7157-474e-81fb-2023ddff7963 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:35.696810 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b649a3-7157-474e-81fb-2023ddff7963 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:35.697082 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b649a3-7157-474e-81fb-2023ddff7963 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:35.703719 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b649a3-7157-474e-81fb-2023ddff7963 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:35.705155 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b649a3-7157-474e-81fb-2023ddff7963 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:35.719522 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-97b649a3-7157-474e-81fb-2023ddff7963 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:53:35.719522 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:53:35.719522 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:53:35.719522 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:53:35.719522 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:53:35.719522 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:53:35.719522 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:53:35.719522 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:53:35.719522 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:53:35.719522 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:53:35.719522 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:53:35.719522 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:53:35.719522 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 21:53:35.719522 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 21:53:35.719522 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 21:53:35.720577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 21:53:35.720577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:53:35.720577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:53:35.720577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:53:35.720577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:53:35.720577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:53:35.720577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:53:35.720577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:53:35.720577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Jun 07 21:53:35.720577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:53:35.720577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:53:35.720577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:53:35.720577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:53:35.720577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:53:35.720577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:53:35.720577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:53:35.720577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:53:35.721691 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 21:53:35.721691 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 21:53:35.721691 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 21:53:35.721691 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 21:53:35.721691 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 21:53:35.721691 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 21:53:35.721691 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 21:53:35.721691 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Jun 07 21:53:35.721691 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 648/1285] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jun 7 21:53:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:53:35.726735 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfbd1e4-b877-4cc4-941b-90c8c04293d1 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:35.780916 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:35.846245 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfbd1e4-b877-4cc4-941b-90c8c04293d1 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:35.884299 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b1425d-fb2b-4ef7-a1f6-b875a6590e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:35.884953 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1425d-fb2b-4ef7-a1f6-b875a6590e25 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:35.885234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1425d-fb2b-4ef7-a1f6-b875a6590e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:35.885511 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1425d-fb2b-4ef7-a1f6-b875a6590e25 None admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:35.887006 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b1425d-fb2b-4ef7-a1f6-b875a6590e25 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:53:35.887316 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b1425d-fb2b-4ef7-a1f6-b875a6590e25 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:35.888772 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b1425d-fb2b-4ef7-a1f6-b875a6590e25 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:35.900318 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 638/1286] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jun 7 21:53:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:53:35.906963 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b649a3-7157-474e-81fb-2023ddff7963 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:36.003974 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b649a3-7157-474e-81fb-2023ddff7963 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:36.038105 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514170c3-f7bc-4780-8515-4d89013bbc4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:36.038750 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514170c3-f7bc-4780-8515-4d89013bbc4b None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:36.039034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514170c3-f7bc-4780-8515-4d89013bbc4b None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:36.039307 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514170c3-f7bc-4780-8515-4d89013bbc4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:36.040167 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514170c3-f7bc-4780-8515-4d89013bbc4b None admin] RBAC: Adding query filter params (role.id=6b483758dbd44eb4a78df2ce4603d601, scope.project.id=0219dbda54274fdc9bdccbd960c30c6b, user.id=02941c6ad2d54e48b1114d9e030c66c7) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:53:36.040481 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514170c3-f7bc-4780-8515-4d89013bbc4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:36.041903 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514170c3-f7bc-4780-8515-4d89013bbc4b None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:36.047766 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 649/1287] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jun 7 21:53:35 2022] GET /identity/v3/role_assignments?user.id=02941c6ad2d54e48b1114d9e030c66c7&scope.project.id=0219dbda54274fdc9bdccbd960c30c6b&role.id=6b483758dbd44eb4a78df2ce4603d601 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:53:38.324990 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f5c86-0e10-4990-ba8c-f79bfd210148 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:38.325568 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f5c86-0e10-4990-ba8c-f79bfd210148 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:38.325568 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f5c86-0e10-4990-ba8c-f79bfd210148 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:38.326899 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 639/1288] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:53:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:53:38.332026 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76351ad8-85ff-4acd-97ed-1be2d50752aa None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:38.332416 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76351ad8-85ff-4acd-97ed-1be2d50752aa None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:38.332582 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76351ad8-85ff-4acd-97ed-1be2d50752aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:38.349907 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-76351ad8-85ff-4acd-97ed-1be2d50752aa None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:53:38.351094 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-76351ad8-85ff-4acd-97ed-1be2d50752aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:53:38.379473 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:38.417613 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 650/1289] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:53:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:53:38.441156 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f5c86-0e10-4990-ba8c-f79bfd210148 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:38.553803 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f5c86-0e10-4990-ba8c-f79bfd210148 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:38.565723 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:38.589826 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96236bd7-718b-4e74-91c3-3cb117b4854f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:38.590557 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96236bd7-718b-4e74-91c3-3cb117b4854f None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:38.590833 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96236bd7-718b-4e74-91c3-3cb117b4854f None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:38.591104 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96236bd7-718b-4e74-91c3-3cb117b4854f None admin] PATH_INFO: `/v3/services/placement` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:38.591776 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96236bd7-718b-4e74-91c3-3cb117b4854f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:38.593125 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96236bd7-718b-4e74-91c3-3cb117b4854f None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:38.605094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-96236bd7-718b-4e74-91c3-3cb117b4854f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 21:53:38.605094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:53:38.605094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 21:53:38.605094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 21:53:38.605094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 21:53:38.605094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 21:53:38.605094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 21:53:38.605094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:53:38.605094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 21:53:38.605094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:53:38.605094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 21:53:38.605094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:53:38.605094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:53:38.605094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:53:38.605094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:53:38.605094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:53:38.605094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:53:38.606331 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:53:38.606331 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:53:38.606331 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:53:38.606331 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:53:38.606331 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:53:38.606331 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:53:38.606331 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 21:53:38.606331 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 21:53:38.606331 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 21:53:38.606331 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 21:53:38.606331 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:53:38.606331 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:53:38.606331 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:53:38.606331 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:53:38.606331 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:53:38.606331 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:53:38.607436 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:53:38.607436 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:53:38.607436 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:53:38.607436 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:53:38.607436 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:53:38.607436 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:53:38.607436 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:53:38.607436 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:53:38.607436 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:53:38.607436 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:53:38.607436 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 21:53:38.607436 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:53:38.607436 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 21:53:38.607436 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:53:38.607436 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 640/1290] 10.222.0.13 () {56 vars in 1117 bytes} [Tue Jun 7 21:53:38 2022] GET /identity/v3/services/placement => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:53:38.612378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76351ad8-85ff-4acd-97ed-1be2d50752aa None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:38.706814 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76351ad8-85ff-4acd-97ed-1be2d50752aa None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:38.739264 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0641fb9-7a7c-4d69-be70-d3f02f30f3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:38.739958 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0641fb9-7a7c-4d69-be70-d3f02f30f3d0 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:38.740240 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0641fb9-7a7c-4d69-be70-d3f02f30f3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:38.740542 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0641fb9-7a7c-4d69-be70-d3f02f30f3d0 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:38.741248 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0641fb9-7a7c-4d69-be70-d3f02f30f3d0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:53:38.741555 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0641fb9-7a7c-4d69-be70-d3f02f30f3d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:38.742919 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0641fb9-7a7c-4d69-be70-d3f02f30f3d0 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:38.748588 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 651/1291] 10.222.0.13 () {56 vars in 1116 bytes} [Tue Jun 7 21:53:38 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 21:53:38.754290 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96236bd7-718b-4e74-91c3-3cb117b4854f None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:38.851719 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96236bd7-718b-4e74-91c3-3cb117b4854f None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:38.884361 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d3e420-9f1f-43c6-b670-5e55216e6971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:38.885012 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3e420-9f1f-43c6-b670-5e55216e6971 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:38.885295 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3e420-9f1f-43c6-b670-5e55216e6971 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:38.885572 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3e420-9f1f-43c6-b670-5e55216e6971 None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:38.886274 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d3e420-9f1f-43c6-b670-5e55216e6971 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:53:38.886582 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d3e420-9f1f-43c6-b670-5e55216e6971 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:38.887971 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d3e420-9f1f-43c6-b670-5e55216e6971 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:38.892286 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 641/1292] 10.222.0.13 () {56 vars in 1116 bytes} [Tue Jun 7 21:53:38 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:53:41.113807 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d3885-9c23-443b-bbb8-649eb5a8d9c1 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:41.114356 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d3885-9c23-443b-bbb8-649eb5a8d9c1 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:41.114356 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d3885-9c23-443b-bbb8-649eb5a8d9c1 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:41.115755 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 652/1293] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:53:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:53:41.121328 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b429d4d9-7c46-48f3-8077-54a8855160f0 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:41.121754 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b429d4d9-7c46-48f3-8077-54a8855160f0 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:41.121866 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b429d4d9-7c46-48f3-8077-54a8855160f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:41.138728 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-b429d4d9-7c46-48f3-8077-54a8855160f0 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:53:41.139920 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-b429d4d9-7c46-48f3-8077-54a8855160f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:53:41.169254 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:41.206687 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 642/1294] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:53:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:53:41.230150 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d3885-9c23-443b-bbb8-649eb5a8d9c1 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:41.259941 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:41.329577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d3885-9c23-443b-bbb8-649eb5a8d9c1 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:41.362907 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c7a502-1341-43f6-b173-2927ca06418a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:41.363598 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c7a502-1341-43f6-b173-2927ca06418a None admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:41.363926 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c7a502-1341-43f6-b173-2927ca06418a None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:41.364150 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c7a502-1341-43f6-b173-2927ca06418a None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:41.364855 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c7a502-1341-43f6-b173-2927ca06418a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:41.366201 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c7a502-1341-43f6-b173-2927ca06418a None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:41.379022 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 653/1295] 10.222.0.13 () {60 vars in 1141 bytes} [Tue Jun 7 21:53:41 2022] POST /identity/v3/services => generated 246 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 21:53:43.626299 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b2b0bc-0720-4bb7-8b9f-b8297a4c3c1d None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:43.626959 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b2b0bc-0720-4bb7-8b9f-b8297a4c3c1d None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:43.626959 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b2b0bc-0720-4bb7-8b9f-b8297a4c3c1d None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:43.628280 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 643/1296] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:53:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:53:43.633313 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ab4d8-d3f0-4fc9-b244-0c1880c86f4a None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:43.633739 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ab4d8-d3f0-4fc9-b244-0c1880c86f4a None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:43.633866 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ab4d8-d3f0-4fc9-b244-0c1880c86f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:43.650812 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-7c4ab4d8-d3f0-4fc9-b244-0c1880c86f4a None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:53:43.652814 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-7c4ab4d8-d3f0-4fc9-b244-0c1880c86f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:53:43.722950 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 654/1297] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:53:43 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:53:43.746097 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b2b0bc-0720-4bb7-8b9f-b8297a4c3c1d None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:43.844510 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b2b0bc-0720-4bb7-8b9f-b8297a4c3c1d None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:43.877531 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2210afea-4663-427a-80d7-1d58f2f46833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:43.878248 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210afea-4663-427a-80d7-1d58f2f46833 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:43.878528 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210afea-4663-427a-80d7-1d58f2f46833 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:43.878801 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210afea-4663-427a-80d7-1d58f2f46833 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:43.879473 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2210afea-4663-427a-80d7-1d58f2f46833 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:43.880827 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2210afea-4663-427a-80d7-1d58f2f46833 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:43.892857 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-2210afea-4663-427a-80d7-1d58f2f46833 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 21:53:43.892857 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:53:43.892857 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 21:53:43.892857 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 21:53:43.892857 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 21:53:43.892857 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 21:53:43.892857 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 21:53:43.892857 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:53:43.892857 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 21:53:43.892857 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:53:43.892857 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 21:53:43.892857 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:53:43.892857 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:53:43.892857 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:53:43.892857 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:53:43.892857 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:53:43.892857 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:53:43.894133 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:53:43.894133 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:53:43.894133 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:53:43.894133 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:53:43.894133 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:53:43.894133 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:53:43.894133 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 21:53:43.894133 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 21:53:43.894133 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 21:53:43.894133 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 21:53:43.894133 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:53:43.894133 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:53:43.894133 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:53:43.894133 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:53:43.894133 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:53:43.894133 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:53:43.895126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:53:43.895126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:53:43.895126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:53:43.895126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:53:43.895126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:53:43.895126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:53:43.895126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:53:43.895126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:53:43.895126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:53:43.895126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:53:43.895126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 21:53:43.895126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:53:43.895126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 21:53:43.895126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:53:43.895126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 644/1298] 10.222.0.13 () {56 vars in 1117 bytes} [Tue Jun 7 21:53:43 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:53:43.900056 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4ab4d8-d3f0-4fc9-b244-0c1880c86f4a None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:43.917689 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:44.001486 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4ab4d8-d3f0-4fc9-b244-0c1880c86f4a None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:44.033806 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b6c23b-e3d7-4ceb-aa25-3eab1a926b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:44.034454 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b6c23b-e3d7-4ceb-aa25-3eab1a926b66 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:44.034733 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b6c23b-e3d7-4ceb-aa25-3eab1a926b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:44.035006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b6c23b-e3d7-4ceb-aa25-3eab1a926b66 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:44.035734 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b6c23b-e3d7-4ceb-aa25-3eab1a926b66 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:53:44.036037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b6c23b-e3d7-4ceb-aa25-3eab1a926b66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:44.037392 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b6c23b-e3d7-4ceb-aa25-3eab1a926b66 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:44.042554 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 655/1299] 10.222.0.13 () {56 vars in 1116 bytes} [Tue Jun 7 21:53:43 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:53:44.048406 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2210afea-4663-427a-80d7-1d58f2f46833 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:44.080387 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:44.148479 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2210afea-4663-427a-80d7-1d58f2f46833 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:44.180680 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8542f981-2e9c-47ec-ba31-4589de4a18e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:44.181322 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542f981-2e9c-47ec-ba31-4589de4a18e5 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:44.181601 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542f981-2e9c-47ec-ba31-4589de4a18e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:44.181874 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542f981-2e9c-47ec-ba31-4589de4a18e5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:44.183366 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8542f981-2e9c-47ec-ba31-4589de4a18e5 None admin] RBAC: Adding query filter params (interface=public, service_id=89752d4deaf14e488f3aa72ddf9cb64d, region_id=RegionOne) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:53:44.183680 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8542f981-2e9c-47ec-ba31-4589de4a18e5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:44.185030 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8542f981-2e9c-47ec-ba31-4589de4a18e5 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:44.189874 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 645/1300] 10.222.0.13 () {56 vars in 1251 bytes} [Tue Jun 7 21:53:44 2022] GET /identity/v3/endpoints?service_id=89752d4deaf14e488f3aa72ddf9cb64d&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:53:44.195490 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b6c23b-e3d7-4ceb-aa25-3eab1a926b66 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:44.273909 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:44.291146 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b6c23b-e3d7-4ceb-aa25-3eab1a926b66 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:44.324079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6700019-3474-4a94-bfe2-26145875beb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:44.324727 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6700019-3474-4a94-bfe2-26145875beb3 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:44.325010 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6700019-3474-4a94-bfe2-26145875beb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:44.325283 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6700019-3474-4a94-bfe2-26145875beb3 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:44.325953 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6700019-3474-4a94-bfe2-26145875beb3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:44.327311 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6700019-3474-4a94-bfe2-26145875beb3 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:44.333973 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 656/1301] 10.222.0.13 () {56 vars in 1087 bytes} [Tue Jun 7 21:53:44 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 21:53:46.617683 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86194654-4c62-4078-be0d-b3f3f1347cff None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:46.618237 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86194654-4c62-4078-be0d-b3f3f1347cff None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:46.618237 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86194654-4c62-4078-be0d-b3f3f1347cff None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:46.619640 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 646/1302] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:53:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:53:46.625861 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08407383-98d1-4379-a9e5-2ce006df850c None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:46.626293 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08407383-98d1-4379-a9e5-2ce006df850c None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:46.626397 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08407383-98d1-4379-a9e5-2ce006df850c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:46.642635 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-08407383-98d1-4379-a9e5-2ce006df850c None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:53:46.643889 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-08407383-98d1-4379-a9e5-2ce006df850c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:53:46.707247 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 657/1303] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:53:46 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 21:53:46.730337 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86194654-4c62-4078-be0d-b3f3f1347cff None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:46.823883 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:46.831717 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86194654-4c62-4078-be0d-b3f3f1347cff None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:46.876240 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ef16ba-942e-471f-98c5-1ab263d03ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:46.876961 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef16ba-942e-471f-98c5-1ab263d03ab5 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:46.877245 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef16ba-942e-471f-98c5-1ab263d03ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:46.877524 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef16ba-942e-471f-98c5-1ab263d03ab5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:46.878180 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ef16ba-942e-471f-98c5-1ab263d03ab5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:46.879573 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ef16ba-942e-471f-98c5-1ab263d03ab5 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:46.891351 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-00ef16ba-942e-471f-98c5-1ab263d03ab5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 21:53:46.891351 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:53:46.891351 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 21:53:46.891351 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 21:53:46.891351 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 21:53:46.891351 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 21:53:46.891351 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 21:53:46.891351 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:53:46.891351 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 21:53:46.891351 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:53:46.891351 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 21:53:46.891351 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:53:46.891351 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 21:53:46.891351 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 21:53:46.891351 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 21:53:46.891351 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 21:53:46.891351 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 21:53:46.892547 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 21:53:46.892547 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 21:53:46.892547 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 21:53:46.892547 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 21:53:46.892547 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 21:53:46.892547 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 21:53:46.892547 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 21:53:46.892547 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 21:53:46.892547 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 21:53:46.892547 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 21:53:46.892547 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 21:53:46.892547 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 21:53:46.892547 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 21:53:46.892547 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 21:53:46.892547 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 21:53:46.892547 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 21:53:46.893422 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 21:53:46.893422 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Jun 07 21:53:46.893422 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 21:53:46.893422 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Jun 07 21:53:46.893422 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 21:53:46.893422 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 21:53:46.893422 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 21:53:46.893422 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Jun 07 21:53:46.893422 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 21:53:46.893422 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Jun 07 21:53:46.893422 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 21:53:46.893422 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 21:53:46.893422 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 21:53:46.893422 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Jun 07 21:53:46.893422 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 647/1304] 10.222.0.13 () {56 vars in 1117 bytes} [Tue Jun 7 21:53:46 2022] GET /identity/v3/services/placement => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 21:53:46.898734 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08407383-98d1-4379-a9e5-2ce006df850c None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:46.971631 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:46.994412 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08407383-98d1-4379-a9e5-2ce006df850c None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:47.026706 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d59a718-c130-433c-835b-28debf77ce97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:47.027369 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59a718-c130-433c-835b-28debf77ce97 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:47.027654 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59a718-c130-433c-835b-28debf77ce97 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:47.027926 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59a718-c130-433c-835b-28debf77ce97 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:47.028618 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d59a718-c130-433c-835b-28debf77ce97 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:53:47.028917 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d59a718-c130-433c-835b-28debf77ce97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:47.030268 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d59a718-c130-433c-835b-28debf77ce97 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:47.035178 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 658/1305] 10.222.0.13 () {56 vars in 1116 bytes} [Tue Jun 7 21:53:46 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:53:47.040854 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ef16ba-942e-471f-98c5-1ab263d03ab5 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:53:47.144605 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ef16ba-942e-471f-98c5-1ab263d03ab5 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:53:47.176796 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a8ff40-2dbc-4dc0-be62-f150b16d54ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:53:47.177439 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8ff40-2dbc-4dc0-be62-f150b16d54ac None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:53:47.177720 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8ff40-2dbc-4dc0-be62-f150b16d54ac None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:53:47.177994 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8ff40-2dbc-4dc0-be62-f150b16d54ac None admin] PATH_INFO: `/v3/endpoints` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:53:47.178718 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a8ff40-2dbc-4dc0-be62-f150b16d54ac None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:53:47.180094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a8ff40-2dbc-4dc0-be62-f150b16d54ac None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:53:47.197197 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 648/1306] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Jun 7 21:53:47 2022] POST /identity/v3/endpoints => generated 331 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 21:53:57.032396 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=30 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:53:57.193843 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:54:51.433375 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6135317d-4da6-4324-8acd-e13c870141c2 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:54:51.434413 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6135317d-4da6-4324-8acd-e13c870141c2 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:54:51.434921 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6135317d-4da6-4324-8acd-e13c870141c2 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:54:51.440652 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 659/1307] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:54:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:54:51.442161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26a368-160d-4a97-af17-e81ae6962f85 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:54:51.442595 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26a368-160d-4a97-af17-e81ae6962f85 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:54:51.442797 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26a368-160d-4a97-af17-e81ae6962f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:54:51.536562 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-0d26a368-160d-4a97-af17-e81ae6962f85 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:54:51.537833 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-0d26a368-160d-4a97-af17-e81ae6962f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:54:51.660149 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 649/1308] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:54:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:54:51.720779 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fd5ebc-bab0-47d7-98a5-ff98c5aceee1 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:54:51.721161 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fd5ebc-bab0-47d7-98a5-ff98c5aceee1 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:54:51.721529 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fd5ebc-bab0-47d7-98a5-ff98c5aceee1 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:54:51.724215 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 660/1309] 10.222.0.13 () {54 vars in 964 bytes} [Tue Jun 7 21:54:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:54:51.733522 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56601543-7b8e-4c3a-9e10-8ee554c29495 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:54:51.733522 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56601543-7b8e-4c3a-9e10-8ee554c29495 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:54:51.733522 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56601543-7b8e-4c3a-9e10-8ee554c29495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:54:51.839019 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-56601543-7b8e-4c3a-9e10-8ee554c29495 None None] MFA Rules not processed for user `10242251e6284217bfb79c67b1567f75`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:54:51.840446 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-56601543-7b8e-4c3a-9e10-8ee554c29495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:54:51.863303 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:54:51.945181 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 650/1310] 10.222.0.13 () {58 vars in 1061 bytes} [Tue Jun 7 21:54:51 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:54:51.954755 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fd5ebc-bab0-47d7-98a5-ff98c5aceee1 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:54:51.956615 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-84fd5ebc-bab0-47d7-98a5-ff98c5aceee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:54:52.081050 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fd5ebc-bab0-47d7-98a5-ff98c5aceee1 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:54:52.171253 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7299cf-e8a9-4e99-ab78-d832685baa3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:54:52.171856 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7299cf-e8a9-4e99-ab78-d832685baa3c service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:54:52.172252 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7299cf-e8a9-4e99-ab78-d832685baa3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:54:52.172528 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7299cf-e8a9-4e99-ab78-d832685baa3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:54:52.174170 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-6c7299cf-e8a9-4e99-ab78-d832685baa3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:54:52.237192 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:54:52.299027 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7299cf-e8a9-4e99-ab78-d832685baa3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:54:52.300792 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7299cf-e8a9-4e99-ab78-d832685baa3c service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:54:52.321671 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 661/1311] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:54:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:54:55.005204 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a7b35-d56f-4fc4-9364-0863731c03dc None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:54:55.005995 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a7b35-d56f-4fc4-9364-0863731c03dc None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:54:55.006286 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a7b35-d56f-4fc4-9364-0863731c03dc None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:54:55.007890 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 651/1312] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:54:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:54:55.013776 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e456b-1930-4359-a9c2-ab61219b2fea None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:54:55.014222 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e456b-1930-4359-a9c2-ab61219b2fea None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:54:55.014430 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e456b-1930-4359-a9c2-ab61219b2fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:54:55.090064 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-436e456b-1930-4359-a9c2-ab61219b2fea None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:54:55.091480 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-436e456b-1930-4359-a9c2-ab61219b2fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:54:55.191726 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 662/1313] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:54:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:54:55.245807 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf25586-53e8-465e-ba70-69dd96468db1 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:54:55.246260 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf25586-53e8-465e-ba70-69dd96468db1 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:54:55.246689 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf25586-53e8-465e-ba70-69dd96468db1 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:54:55.248404 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 652/1314] 10.222.0.13 () {54 vars in 964 bytes} [Tue Jun 7 21:54:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:54:55.254716 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48343f-960b-4a5f-8cd5-adba5592733a None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:54:55.255007 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48343f-960b-4a5f-8cd5-adba5592733a None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:54:55.255291 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48343f-960b-4a5f-8cd5-adba5592733a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:54:55.266844 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:54:55.339951 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-7a48343f-960b-4a5f-8cd5-adba5592733a None None] MFA Rules not processed for user `10242251e6284217bfb79c67b1567f75`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:54:55.341103 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-7a48343f-960b-4a5f-8cd5-adba5592733a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:54:55.421900 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 663/1315] 10.222.0.13 () {58 vars in 1061 bytes} [Tue Jun 7 21:54:55 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:54:55.431281 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf25586-53e8-465e-ba70-69dd96468db1 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:54:55.432816 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-1cf25586-53e8-465e-ba70-69dd96468db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:54:55.539718 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf25586-53e8-465e-ba70-69dd96468db1 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:54:55.613802 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:54:55.616664 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db635194-42ee-4bcf-b00a-fadec9133b4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:54:55.617325 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db635194-42ee-4bcf-b00a-fadec9133b4b service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:54:55.617947 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db635194-42ee-4bcf-b00a-fadec9133b4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:54:55.618287 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db635194-42ee-4bcf-b00a-fadec9133b4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:54:55.620370 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-db635194-42ee-4bcf-b00a-fadec9133b4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:54:55.735429 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db635194-42ee-4bcf-b00a-fadec9133b4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:54:55.736875 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db635194-42ee-4bcf-b00a-fadec9133b4b service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:54:55.756415 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 653/1316] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:54:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:54:58.591839 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04316233-4b79-49c0-be0e-1340d2eb32c3 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:54:58.592403 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04316233-4b79-49c0-be0e-1340d2eb32c3 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:54:58.592403 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04316233-4b79-49c0-be0e-1340d2eb32c3 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:54:58.593761 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 664/1317] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:54:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:54:58.599177 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7e1fc-39a2-4dbe-b236-c0f4c9778e41 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:54:58.599579 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7e1fc-39a2-4dbe-b236-c0f4c9778e41 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:54:58.599779 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7e1fc-39a2-4dbe-b236-c0f4c9778e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:54:58.793880 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-6dd7e1fc-39a2-4dbe-b236-c0f4c9778e41 None None] MFA Rules not processed for user `9b2edac556674677ba6894dc26bd2aab`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:54:58.795108 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-6dd7e1fc-39a2-4dbe-b236-c0f4c9778e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:54:58.846182 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:54:58.890053 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 654/1318] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:54:58 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:54:58.929677 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04316233-4b79-49c0-be0e-1340d2eb32c3 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:54:58.931592 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-04316233-4b79-49c0-be0e-1340d2eb32c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:54:59.033538 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04316233-4b79-49c0-be0e-1340d2eb32c3 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:54:59.036384 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:54:59.112826 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f9f2cf-0153-4ca0-a372-b5d2a731ed20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:54:59.113393 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f9f2cf-0153-4ca0-a372-b5d2a731ed20 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:54:59.113675 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f9f2cf-0153-4ca0-a372-b5d2a731ed20 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:54:59.113951 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f9f2cf-0153-4ca0-a372-b5d2a731ed20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:54:59.115617 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-f5f9f2cf-0153-4ca0-a372-b5d2a731ed20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:54:59.221163 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f9f2cf-0153-4ca0-a372-b5d2a731ed20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:54:59.222838 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f9f2cf-0153-4ca0-a372-b5d2a731ed20 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:54:59.240833 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 665/1319] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:54:58 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:55:02.081226 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf07cd-4a88-4838-a2e9-eb8f028ab952 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:02.081739 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf07cd-4a88-4838-a2e9-eb8f028ab952 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:02.081827 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf07cd-4a88-4838-a2e9-eb8f028ab952 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:02.083134 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 655/1320] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:55:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:55:02.089041 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36870524-3664-40a0-a1ff-b1db07179f88 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:02.089475 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36870524-3664-40a0-a1ff-b1db07179f88 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:02.089571 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36870524-3664-40a0-a1ff-b1db07179f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:02.131773 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:55:02.159127 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-36870524-3664-40a0-a1ff-b1db07179f88 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:55:02.160354 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-36870524-3664-40a0-a1ff-b1db07179f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:55:02.259193 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 666/1321] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:55:02 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:55:02.297521 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf07cd-4a88-4838-a2e9-eb8f028ab952 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:55:02.299030 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-fdbf07cd-4a88-4838-a2e9-eb8f028ab952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:55:02.400692 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf07cd-4a88-4838-a2e9-eb8f028ab952 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:55:02.472359 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e927cf00-2a67-418c-a887-da6dc7f5f04a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:55:02.473261 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e927cf00-2a67-418c-a887-da6dc7f5f04a service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:02.473542 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e927cf00-2a67-418c-a887-da6dc7f5f04a service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:02.473816 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e927cf00-2a67-418c-a887-da6dc7f5f04a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:02.475448 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-e927cf00-2a67-418c-a887-da6dc7f5f04a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:55:02.482930 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:55:02.582611 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e927cf00-2a67-418c-a887-da6dc7f5f04a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:55:02.584059 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e927cf00-2a67-418c-a887-da6dc7f5f04a service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:55:02.602454 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 656/1322] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:55:02 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:55:05.506151 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e8a05-c337-4c52-a25b-533008e89604 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:05.506695 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e8a05-c337-4c52-a25b-533008e89604 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:05.506695 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e8a05-c337-4c52-a25b-533008e89604 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:05.508838 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 667/1323] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:55:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:55:05.518368 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd4970a-c984-48e3-9463-aa0b3b52c9e5 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:05.518835 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd4970a-c984-48e3-9463-aa0b3b52c9e5 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:05.519014 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd4970a-c984-48e3-9463-aa0b3b52c9e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:05.584637 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-dcd4970a-c984-48e3-9463-aa0b3b52c9e5 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:55:05.585814 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-dcd4970a-c984-48e3-9463-aa0b3b52c9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:55:05.644091 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:55:05.682630 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 657/1324] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:55:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:55:05.719569 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e8a05-c337-4c52-a25b-533008e89604 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:55:05.721078 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-1f8e8a05-c337-4c52-a25b-533008e89604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:55:05.833394 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e8a05-c337-4c52-a25b-533008e89604 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:55:05.846317 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:55:05.908829 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf603d38-ae33-49af-b6cc-e784650b6d0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:55:05.909475 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf603d38-ae33-49af-b6cc-e784650b6d0c service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:05.909760 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf603d38-ae33-49af-b6cc-e784650b6d0c service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:05.910036 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf603d38-ae33-49af-b6cc-e784650b6d0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:05.911708 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-bf603d38-ae33-49af-b6cc-e784650b6d0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:55:06.025358 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf603d38-ae33-49af-b6cc-e784650b6d0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:55:06.026716 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf603d38-ae33-49af-b6cc-e784650b6d0c service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:55:06.045589 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 668/1325] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:55:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:55:08.365173 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece767e-68f1-468d-b13e-4e99aaaf70b4 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:08.365738 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece767e-68f1-468d-b13e-4e99aaaf70b4 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:08.365738 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece767e-68f1-468d-b13e-4e99aaaf70b4 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:08.367072 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 658/1326] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:55:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:55:08.374689 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b162880-b424-471b-8a2f-77611c295c35 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:08.375139 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b162880-b424-471b-8a2f-77611c295c35 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:08.375377 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b162880-b424-471b-8a2f-77611c295c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:08.426348 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:55:08.446522 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-6b162880-b424-471b-8a2f-77611c295c35 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:55:08.447741 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-6b162880-b424-471b-8a2f-77611c295c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:55:08.546857 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 669/1327] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:55:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:55:08.582881 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece767e-68f1-468d-b13e-4e99aaaf70b4 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:55:08.584441 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-bece767e-68f1-468d-b13e-4e99aaaf70b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:55:08.687649 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece767e-68f1-468d-b13e-4e99aaaf70b4 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:55:08.762121 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ccdea3-d1e0-4581-baa9-11841c287d33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:55:08.762789 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ccdea3-d1e0-4581-baa9-11841c287d33 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:08.763077 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ccdea3-d1e0-4581-baa9-11841c287d33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:08.763389 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ccdea3-d1e0-4581-baa9-11841c287d33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:08.765057 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-78ccdea3-d1e0-4581-baa9-11841c287d33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:55:08.782174 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:55:08.879814 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ccdea3-d1e0-4581-baa9-11841c287d33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:55:08.881266 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ccdea3-d1e0-4581-baa9-11841c287d33 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:55:08.902378 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 659/1328] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:55:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 21:55:11.222566 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40782e-82d3-41e7-84f9-1da6d28bb33f None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:11.223349 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40782e-82d3-41e7-84f9-1da6d28bb33f None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:11.223349 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40782e-82d3-41e7-84f9-1da6d28bb33f None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:11.224709 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 670/1329] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:55:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:55:11.230482 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff4ae02-78d1-49c7-b960-b683f24f2eef None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:11.230886 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff4ae02-78d1-49c7-b960-b683f24f2eef None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:11.231045 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff4ae02-78d1-49c7-b960-b683f24f2eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:11.295680 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-aff4ae02-78d1-49c7-b960-b683f24f2eef None None] MFA Rules not processed for user `9b2edac556674677ba6894dc26bd2aab`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:55:11.296896 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-aff4ae02-78d1-49c7-b960-b683f24f2eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:55:11.362041 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:55:11.394549 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 660/1330] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:55:11 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:55:11.432708 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40782e-82d3-41e7-84f9-1da6d28bb33f None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:55:11.434300 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-9a40782e-82d3-41e7-84f9-1da6d28bb33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:55:11.538044 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40782e-82d3-41e7-84f9-1da6d28bb33f None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:55:11.558289 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:55:11.615607 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746278ce-0858-4df4-a9c7-e81362d6937e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:55:11.616252 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746278ce-0858-4df4-a9c7-e81362d6937e service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:11.616537 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746278ce-0858-4df4-a9c7-e81362d6937e service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:11.616815 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746278ce-0858-4df4-a9c7-e81362d6937e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:11.618504 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-746278ce-0858-4df4-a9c7-e81362d6937e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:55:11.726026 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746278ce-0858-4df4-a9c7-e81362d6937e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:55:11.727774 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746278ce-0858-4df4-a9c7-e81362d6937e service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:55:11.746280 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 671/1331] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:55:11 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:55:14.391732 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df12cc-f1cf-42b0-ae1e-187c0a15ec28 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:14.392235 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df12cc-f1cf-42b0-ae1e-187c0a15ec28 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:14.392328 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df12cc-f1cf-42b0-ae1e-187c0a15ec28 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:14.400030 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 661/1332] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:55:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:55:14.400141 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7e8d5c-780d-4f6a-ac5d-88a518f8d634 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:14.400485 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7e8d5c-780d-4f6a-ac5d-88a518f8d634 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:14.400699 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7e8d5c-780d-4f6a-ac5d-88a518f8d634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:14.471791 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:55:14.475511 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-2e7e8d5c-780d-4f6a-ac5d-88a518f8d634 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:55:14.477513 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-2e7e8d5c-780d-4f6a-ac5d-88a518f8d634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:55:14.589443 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 672/1333] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:55:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:55:14.625574 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3df12cc-f1cf-42b0-ae1e-187c0a15ec28 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:55:14.627079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-c3df12cc-f1cf-42b0-ae1e-187c0a15ec28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:55:14.730536 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3df12cc-f1cf-42b0-ae1e-187c0a15ec28 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:55:14.808334 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2afb216-a05c-4d13-b938-0dcff962123b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:55:14.808985 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2afb216-a05c-4d13-b938-0dcff962123b service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:14.809265 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2afb216-a05c-4d13-b938-0dcff962123b service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:14.809539 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2afb216-a05c-4d13-b938-0dcff962123b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:14.811163 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-d2afb216-a05c-4d13-b938-0dcff962123b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:55:14.838660 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:55:14.927959 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2afb216-a05c-4d13-b938-0dcff962123b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:55:14.929396 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2afb216-a05c-4d13-b938-0dcff962123b service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:55:14.947830 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 662/1334] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:55:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:55:17.290174 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ccc3c7-5296-4838-8bfa-671812d075d3 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:17.290882 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ccc3c7-5296-4838-8bfa-671812d075d3 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:17.291128 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ccc3c7-5296-4838-8bfa-671812d075d3 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:17.292551 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 673/1335] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:55:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:55:17.297853 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36291bea-92c3-4124-8521-48b8dad85d5f None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:17.298290 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36291bea-92c3-4124-8521-48b8dad85d5f None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:17.298398 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36291bea-92c3-4124-8521-48b8dad85d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:17.362815 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-36291bea-92c3-4124-8521-48b8dad85d5f None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:55:17.364037 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-36291bea-92c3-4124-8521-48b8dad85d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:55:17.439181 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:55:17.461008 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 663/1336] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:55:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:55:17.501475 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ccc3c7-5296-4838-8bfa-671812d075d3 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:55:17.502985 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-f7ccc3c7-5296-4838-8bfa-671812d075d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:55:17.605495 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ccc3c7-5296-4838-8bfa-671812d075d3 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:55:17.632353 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:55:17.680682 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d3c165-f495-4a22-abac-7f489cc0a89a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:55:17.681330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3c165-f495-4a22-abac-7f489cc0a89a service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:17.681608 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3c165-f495-4a22-abac-7f489cc0a89a service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:17.681874 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3c165-f495-4a22-abac-7f489cc0a89a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:17.683518 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-f1d3c165-f495-4a22-abac-7f489cc0a89a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:55:17.799673 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d3c165-f495-4a22-abac-7f489cc0a89a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:55:17.801123 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d3c165-f495-4a22-abac-7f489cc0a89a service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:55:17.819580 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 674/1337] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:55:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:55:20.414935 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38daff10-02a8-4d4d-82f9-f514a18b2b8b None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:20.415522 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38daff10-02a8-4d4d-82f9-f514a18b2b8b None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:20.415522 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38daff10-02a8-4d4d-82f9-f514a18b2b8b None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:20.416893 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 664/1338] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:55:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:55:20.422178 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4dba4-ec03-4156-a9b7-f83f535c4c5d None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:20.422597 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4dba4-ec03-4156-a9b7-f83f535c4c5d None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:20.422744 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4dba4-ec03-4156-a9b7-f83f535c4c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:20.486510 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-a2e4dba4-ec03-4156-a9b7-f83f535c4c5d None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:55:20.487743 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a2e4dba4-ec03-4156-a9b7-f83f535c4c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:55:20.498728 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:55:20.586303 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 675/1339] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:55:20 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:55:20.622950 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38daff10-02a8-4d4d-82f9-f514a18b2b8b None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:55:20.624491 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-38daff10-02a8-4d4d-82f9-f514a18b2b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:55:20.727221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38daff10-02a8-4d4d-82f9-f514a18b2b8b None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:55:20.803810 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c253ac-da7c-443a-a4ea-813d0d45c5cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:55:20.804585 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c253ac-da7c-443a-a4ea-813d0d45c5cc service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:20.804877 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c253ac-da7c-443a-a4ea-813d0d45c5cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:20.805160 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c253ac-da7c-443a-a4ea-813d0d45c5cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:20.806839 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-d2c253ac-da7c-443a-a4ea-813d0d45c5cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:55:20.843192 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:55:20.917068 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c253ac-da7c-443a-a4ea-813d0d45c5cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:55:20.918528 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c253ac-da7c-443a-a4ea-813d0d45c5cc service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:55:20.936064 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 665/1340] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:55:20 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:55:25.217566 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5f6cb-1057-4dc8-89c7-c71409b40fdd None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:25.218092 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5f6cb-1057-4dc8-89c7-c71409b40fdd None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:25.218092 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5f6cb-1057-4dc8-89c7-c71409b40fdd None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:25.219509 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 676/1341] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:55:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:55:25.225079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e277e-9d2f-4daf-9fd9-0880bbc51a46 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:25.225589 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e277e-9d2f-4daf-9fd9-0880bbc51a46 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:25.225769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e277e-9d2f-4daf-9fd9-0880bbc51a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:25.297312 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-c97e277e-9d2f-4daf-9fd9-0880bbc51a46 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:55:25.298513 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-c97e277e-9d2f-4daf-9fd9-0880bbc51a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:55:25.396089 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:55:25.410303 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 666/1342] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:55:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:55:25.449591 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df5f6cb-1057-4dc8-89c7-c71409b40fdd None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:55:25.451740 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-7df5f6cb-1057-4dc8-89c7-c71409b40fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:55:25.550816 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df5f6cb-1057-4dc8-89c7-c71409b40fdd None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:55:25.554759 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a28a3ad-0e04-48d1-8006-1a7613077cc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:55:25.555258 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a28a3ad-0e04-48d1-8006-1a7613077cc3 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:25.555633 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a28a3ad-0e04-48d1-8006-1a7613077cc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:25.555904 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a28a3ad-0e04-48d1-8006-1a7613077cc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:25.565200 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a28a3ad-0e04-48d1-8006-1a7613077cc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:55:25.566850 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a28a3ad-0e04-48d1-8006-1a7613077cc3 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:55:25.577168 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 677/1343] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:55:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:55:28.460308 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb1ce74-41c8-46dd-8aea-f5d92bee355c None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:28.460865 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb1ce74-41c8-46dd-8aea-f5d92bee355c None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:28.460865 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb1ce74-41c8-46dd-8aea-f5d92bee355c None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:28.462163 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 667/1344] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:55:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:55:28.467290 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af07213-9323-4edd-87c2-e29e1333a50c None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:28.467717 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af07213-9323-4edd-87c2-e29e1333a50c None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:28.467879 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af07213-9323-4edd-87c2-e29e1333a50c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:28.497888 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-7af07213-9323-4edd-87c2-e29e1333a50c None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:55:28.499292 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-7af07213-9323-4edd-87c2-e29e1333a50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:55:28.531803 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:55:28.582537 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 678/1345] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:55:28 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:55:28.620135 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb1ce74-41c8-46dd-8aea-f5d92bee355c None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:55:28.630640 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb1ce74-41c8-46dd-8aea-f5d92bee355c None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:55:28.634257 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd2e28-977c-4453-a6a3-e23441d84dab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:55:28.634898 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd2e28-977c-4453-a6a3-e23441d84dab service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:28.635184 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd2e28-977c-4453-a6a3-e23441d84dab service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:28.635499 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd2e28-977c-4453-a6a3-e23441d84dab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:28.644424 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd2e28-977c-4453-a6a3-e23441d84dab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:55:28.645850 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd2e28-977c-4453-a6a3-e23441d84dab service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:55:28.655368 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 668/1346] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:55:28 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:55:32.836346 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da6304-0d69-43d7-aae7-248dbcf04e5b None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:32.836884 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da6304-0d69-43d7-aae7-248dbcf04e5b None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:32.836884 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da6304-0d69-43d7-aae7-248dbcf04e5b None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:32.838303 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 679/1347] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:55:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:55:32.843653 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3f882-7d12-488b-9ece-456d9b462e76 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:32.844052 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3f882-7d12-488b-9ece-456d9b462e76 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:32.844234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3f882-7d12-488b-9ece-456d9b462e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:32.862503 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-d7e3f882-7d12-488b-9ece-456d9b462e76 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:55:32.863713 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-d7e3f882-7d12-488b-9ece-456d9b462e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:55:32.872484 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 669/1348] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:55:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:55:32.910201 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71da6304-0d69-43d7-aae7-248dbcf04e5b None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:55:32.921554 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71da6304-0d69-43d7-aae7-248dbcf04e5b None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:55:32.925052 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1019599-08d9-4b2f-a002-11c865eb82da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:55:32.925687 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1019599-08d9-4b2f-a002-11c865eb82da service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:32.925968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1019599-08d9-4b2f-a002-11c865eb82da service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:32.926248 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1019599-08d9-4b2f-a002-11c865eb82da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:32.936487 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1019599-08d9-4b2f-a002-11c865eb82da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:55:32.937845 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1019599-08d9-4b2f-a002-11c865eb82da service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:55:32.947453 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 680/1349] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:55:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:55:38.511679 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd804c-7f16-4440-87e5-0931d433145d None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:38.512218 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd804c-7f16-4440-87e5-0931d433145d None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:38.512218 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd804c-7f16-4440-87e5-0931d433145d None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:38.513839 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 670/1350] 10.222.0.13 () {54 vars in 921 bytes} [Tue Jun 7 21:55:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:55:38.519095 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6eb3e0-bed0-4613-b26c-422bef641d70 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:38.519919 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6eb3e0-bed0-4613-b26c-422bef641d70 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:38.520425 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6eb3e0-bed0-4613-b26c-422bef641d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:38.637602 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-4c6eb3e0-bed0-4613-b26c-422bef641d70 None None] MFA Rules not processed for user `02941c6ad2d54e48b1114d9e030c66c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:55:38.639617 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-4c6eb3e0-bed0-4613-b26c-422bef641d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:55:38.758066 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 681/1351] 10.222.0.13 () {58 vars in 1018 bytes} [Tue Jun 7 21:55:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:55:42.857479 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=13 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:55:43.038856 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80adec88-b35e-45fa-bcf5-8f4cd1cd5a23 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:43.039547 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80adec88-b35e-45fa-bcf5-8f4cd1cd5a23 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:43.039833 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80adec88-b35e-45fa-bcf5-8f4cd1cd5a23 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:43.041409 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 671/1352] 10.222.0.13 () {54 vars in 921 bytes} [Tue Jun 7 21:55:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:55:43.046294 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc3f51-85a8-47c5-bf02-5acff2dddac0 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:43.046782 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc3f51-85a8-47c5-bf02-5acff2dddac0 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:43.047060 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc3f51-85a8-47c5-bf02-5acff2dddac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:43.067099 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-edfc3f51-85a8-47c5-bf02-5acff2dddac0 None None] MFA Rules not processed for user `02941c6ad2d54e48b1114d9e030c66c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:55:43.068474 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-edfc3f51-85a8-47c5-bf02-5acff2dddac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:55:43.077774 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 682/1353] 10.222.0.13 () {58 vars in 1018 bytes} [Tue Jun 7 21:55:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:55:44.677812 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9d58e-60c0-45b9-8484-01705ea49a93 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:44.678385 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9d58e-60c0-45b9-8484-01705ea49a93 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:44.678385 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9d58e-60c0-45b9-8484-01705ea49a93 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:44.680403 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 672/1354] 10.222.0.13 () {54 vars in 921 bytes} [Tue Jun 7 21:55:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:55:44.685522 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c53eb-09e1-4006-94d3-654f75873845 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:44.685976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c53eb-09e1-4006-94d3-654f75873845 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:44.686185 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c53eb-09e1-4006-94d3-654f75873845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:44.705185 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-9b9c53eb-09e1-4006-94d3-654f75873845 None None] MFA Rules not processed for user `02941c6ad2d54e48b1114d9e030c66c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:55:44.706375 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-9b9c53eb-09e1-4006-94d3-654f75873845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:55:44.715104 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 683/1355] 10.222.0.13 () {58 vars in 1018 bytes} [Tue Jun 7 21:55:44 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:55:48.624423 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b795d-27f4-438c-98ed-03dc08510ceb None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:48.625025 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b795d-27f4-438c-98ed-03dc08510ceb None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:48.625025 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b795d-27f4-438c-98ed-03dc08510ceb None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:48.626497 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 673/1356] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:55:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:55:48.631589 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4706d9c7-16e6-47d0-b3ef-18bb3ad54361 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:48.632698 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4706d9c7-16e6-47d0-b3ef-18bb3ad54361 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:48.632880 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4706d9c7-16e6-47d0-b3ef-18bb3ad54361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:48.651338 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-4706d9c7-16e6-47d0-b3ef-18bb3ad54361 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:55:48.652682 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-4706d9c7-16e6-47d0-b3ef-18bb3ad54361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:55:48.661270 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 684/1357] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:55:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:55:48.723701 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25693492-406a-47d4-84bd-682ebc00f3ed None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:48.723991 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25693492-406a-47d4-84bd-682ebc00f3ed None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:48.724462 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25693492-406a-47d4-84bd-682ebc00f3ed None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:48.726756 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 674/1358] 10.222.0.13 () {54 vars in 961 bytes} [Tue Jun 7 21:55:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:55:48.737084 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc36e71-5644-4eca-a9d6-4e5c2df2ea8d None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:48.737489 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc36e71-5644-4eca-a9d6-4e5c2df2ea8d None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:48.738087 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc36e71-5644-4eca-a9d6-4e5c2df2ea8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:48.799087 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:55:48.802585 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-7dc36e71-5644-4eca-a9d6-4e5c2df2ea8d None None] MFA Rules not processed for user `dbe0303ea0764347bfb054870ebba41f`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:55:48.804882 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-7dc36e71-5644-4eca-a9d6-4e5c2df2ea8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:55:48.898731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 685/1359] 10.222.0.13 () {58 vars in 1058 bytes} [Tue Jun 7 21:55:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:55:48.911723 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25693492-406a-47d4-84bd-682ebc00f3ed None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:55:48.932945 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25693492-406a-47d4-84bd-682ebc00f3ed None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:55:48.939691 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e5ab3e-460b-4b51-b396-7c32f367f235 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:55:48.940463 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5ab3e-460b-4b51-b396-7c32f367f235 service nova] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:48.940846 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5ab3e-460b-4b51-b396-7c32f367f235 service nova] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:48.941221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5ab3e-460b-4b51-b396-7c32f367f235 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:48.952523 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e5ab3e-460b-4b51-b396-7c32f367f235 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:55:48.955003 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e5ab3e-460b-4b51-b396-7c32f367f235 service nova] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:55:48.975552 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 675/1360] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:55:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:55:51.511562 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f2dca-f187-4c16-9c79-bfaa9c9ecfcd None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:51.512011 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f2dca-f187-4c16-9c79-bfaa9c9ecfcd None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:51.512120 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f2dca-f187-4c16-9c79-bfaa9c9ecfcd None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:51.513636 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 686/1361] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:55:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:55:51.519137 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c34f89-cd44-4844-889e-562eea2138fe None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:51.519620 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c34f89-cd44-4844-889e-562eea2138fe None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:51.519740 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c34f89-cd44-4844-889e-562eea2138fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:51.537863 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-55c34f89-cd44-4844-889e-562eea2138fe None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:55:51.539353 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-55c34f89-cd44-4844-889e-562eea2138fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:55:51.547793 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 676/1362] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:55:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:55:51.591844 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f2dca-f187-4c16-9c79-bfaa9c9ecfcd None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:55:51.608090 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f2dca-f187-4c16-9c79-bfaa9c9ecfcd None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:55:51.613918 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e6cb7-cb60-4673-8eca-1115c274ac14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:55:51.614774 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e6cb7-cb60-4673-8eca-1115c274ac14 service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:51.615268 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e6cb7-cb60-4673-8eca-1115c274ac14 service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:51.615866 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e6cb7-cb60-4673-8eca-1115c274ac14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:51.625972 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1e6cb7-cb60-4673-8eca-1115c274ac14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:55:51.627864 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1e6cb7-cb60-4673-8eca-1115c274ac14 service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:55:51.641275 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 687/1363] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:55:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:55:54.290943 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ad0ae-e102-4b54-94e6-f0a6d5cff327 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:54.290943 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ad0ae-e102-4b54-94e6-f0a6d5cff327 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:54.290943 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ad0ae-e102-4b54-94e6-f0a6d5cff327 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:54.292196 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 677/1364] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:55:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:55:54.298429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b6ea5-88d1-40ce-a315-99641fd443f6 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:54.299608 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b6ea5-88d1-40ce-a315-99641fd443f6 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:54.300418 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b6ea5-88d1-40ce-a315-99641fd443f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:54.327784 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-c23b6ea5-88d1-40ce-a315-99641fd443f6 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:55:54.329012 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c23b6ea5-88d1-40ce-a315-99641fd443f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:55:54.337435 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 688/1365] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:55:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:55:54.383234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8067d4dd-3896-49ec-b6d3-56e77c3eaee1 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:54.383546 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8067d4dd-3896-49ec-b6d3-56e77c3eaee1 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:54.383824 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8067d4dd-3896-49ec-b6d3-56e77c3eaee1 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:54.385176 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 678/1366] 10.222.0.13 () {54 vars in 961 bytes} [Tue Jun 7 21:55:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:55:54.390653 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd08a4-7229-4d39-be7a-c20735250aa4 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:54.390944 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd08a4-7229-4d39-be7a-c20735250aa4 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:54.391215 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd08a4-7229-4d39-be7a-c20735250aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:54.409976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-b7dd08a4-7229-4d39-be7a-c20735250aa4 None None] MFA Rules not processed for user `dbe0303ea0764347bfb054870ebba41f`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:55:54.411142 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-b7dd08a4-7229-4d39-be7a-c20735250aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:55:54.420982 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 689/1367] 10.222.0.13 () {58 vars in 1058 bytes} [Tue Jun 7 21:55:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:55:54.428638 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8067d4dd-3896-49ec-b6d3-56e77c3eaee1 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:55:54.442663 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8067d4dd-3896-49ec-b6d3-56e77c3eaee1 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:55:54.448040 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f90bc2a-e690-4af6-8ea5-9c2d8ddf4417 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:55:54.448694 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f90bc2a-e690-4af6-8ea5-9c2d8ddf4417 service nova] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:54.448975 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f90bc2a-e690-4af6-8ea5-9c2d8ddf4417 service nova] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:54.449247 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f90bc2a-e690-4af6-8ea5-9c2d8ddf4417 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:54.457104 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f90bc2a-e690-4af6-8ea5-9c2d8ddf4417 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:55:54.458653 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f90bc2a-e690-4af6-8ea5-9c2d8ddf4417 service nova] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:55:54.468441 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 679/1368] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:55:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:55:56.812276 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56ae08-f8b1-43ee-9c74-e8af013275f1 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:56.812799 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56ae08-f8b1-43ee-9c74-e8af013275f1 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:56.812916 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56ae08-f8b1-43ee-9c74-e8af013275f1 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:56.814381 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 690/1369] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:55:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:55:56.819516 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb19b5c-dc4e-481f-b723-1e463639d67f None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:56.819903 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb19b5c-dc4e-481f-b723-1e463639d67f None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:56.820074 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb19b5c-dc4e-481f-b723-1e463639d67f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:56.839722 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-ceb19b5c-dc4e-481f-b723-1e463639d67f None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:55:56.841146 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-ceb19b5c-dc4e-481f-b723-1e463639d67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:55:56.849421 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 680/1370] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:55:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:55:56.894020 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f56ae08-f8b1-43ee-9c74-e8af013275f1 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:55:56.909098 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f56ae08-f8b1-43ee-9c74-e8af013275f1 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:55:56.914683 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8936cd-1dc7-4870-84a3-faa70a9821a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:55:56.915353 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8936cd-1dc7-4870-84a3-faa70a9821a4 service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:56.915640 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8936cd-1dc7-4870-84a3-faa70a9821a4 service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:56.915915 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8936cd-1dc7-4870-84a3-faa70a9821a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:56.925108 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8936cd-1dc7-4870-84a3-faa70a9821a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:55:56.926728 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8936cd-1dc7-4870-84a3-faa70a9821a4 service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:55:56.937056 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 691/1371] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:55:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:55:57.118608 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8a9cb-4b8c-43bc-b66f-9d6e65d89065 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:57.118875 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8a9cb-4b8c-43bc-b66f-9d6e65d89065 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:57.119153 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8a9cb-4b8c-43bc-b66f-9d6e65d89065 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:57.120807 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 681/1372] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:55:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:55:57.127286 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbedf600-3803-4112-b75d-1bc5344a7ea8 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:57.127628 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbedf600-3803-4112-b75d-1bc5344a7ea8 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:57.127912 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbedf600-3803-4112-b75d-1bc5344a7ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:57.148815 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-fbedf600-3803-4112-b75d-1bc5344a7ea8 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:55:57.149781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-fbedf600-3803-4112-b75d-1bc5344a7ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:55:57.159540 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 692/1373] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:55:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:55:57.465362 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ded5dd-d7de-4d79-8c3c-138f63976933 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:57.465648 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ded5dd-d7de-4d79-8c3c-138f63976933 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:57.465933 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ded5dd-d7de-4d79-8c3c-138f63976933 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:57.467530 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 682/1374] 10.222.0.13 () {54 vars in 963 bytes} [Tue Jun 7 21:55:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:55:57.472515 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b501aa27-c017-437d-8b1a-e0d4b1a4ac35 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:57.472896 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b501aa27-c017-437d-8b1a-e0d4b1a4ac35 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:57.473503 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b501aa27-c017-437d-8b1a-e0d4b1a4ac35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:57.533233 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-b501aa27-c017-437d-8b1a-e0d4b1a4ac35 None None] MFA Rules not processed for user `ea8a1e2cf1da4f87b8d4e16f525fc759`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:55:57.534502 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-b501aa27-c017-437d-8b1a-e0d4b1a4ac35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:55:57.600043 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 693/1375] 10.222.0.13 () {58 vars in 1060 bytes} [Tue Jun 7 21:55:57 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:55:57.607804 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ded5dd-d7de-4d79-8c3c-138f63976933 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:55:57.618585 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ded5dd-d7de-4d79-8c3c-138f63976933 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:55:57.622347 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db1c6f0-741a-4a7b-b1cf-b0b2e4c68d38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:55:57.623078 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1c6f0-741a-4a7b-b1cf-b0b2e4c68d38 service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:57.623462 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1c6f0-741a-4a7b-b1cf-b0b2e4c68d38 service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:57.623809 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1c6f0-741a-4a7b-b1cf-b0b2e4c68d38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:57.633385 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db1c6f0-741a-4a7b-b1cf-b0b2e4c68d38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:55:57.634873 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db1c6f0-741a-4a7b-b1cf-b0b2e4c68d38 service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:55:57.644651 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 683/1376] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:55:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:55:59.243922 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64056b-0065-45d8-96fa-8b02f91fe4af None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:59.244419 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64056b-0065-45d8-96fa-8b02f91fe4af None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:59.244513 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64056b-0065-45d8-96fa-8b02f91fe4af None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:59.245877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 694/1377] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:55:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:55:59.251241 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a87edee-716e-4f25-9ce1-aaf301c7a494 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:59.251651 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a87edee-716e-4f25-9ce1-aaf301c7a494 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:59.251839 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a87edee-716e-4f25-9ce1-aaf301c7a494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:59.269189 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-3a87edee-716e-4f25-9ce1-aaf301c7a494 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:55:59.270415 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-3a87edee-716e-4f25-9ce1-aaf301c7a494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:55:59.278181 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 684/1378] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:55:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:55:59.322336 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb64056b-0065-45d8-96fa-8b02f91fe4af None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:55:59.336925 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb64056b-0065-45d8-96fa-8b02f91fe4af None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:55:59.344043 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c97332-57c8-4d1d-8fce-b14c9aa2dc5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:55:59.344685 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c97332-57c8-4d1d-8fce-b14c9aa2dc5e service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:55:59.344971 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c97332-57c8-4d1d-8fce-b14c9aa2dc5e service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:55:59.345251 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c97332-57c8-4d1d-8fce-b14c9aa2dc5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:55:59.355783 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c97332-57c8-4d1d-8fce-b14c9aa2dc5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:55:59.357427 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c97332-57c8-4d1d-8fce-b14c9aa2dc5e service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:55:59.361911 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:55:59.369669 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 695/1379] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:55:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:56:01.717519 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ee1df-8f2f-478d-9e2e-5f8d1511ccb9 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:01.718663 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ee1df-8f2f-478d-9e2e-5f8d1511ccb9 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:01.719642 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ee1df-8f2f-478d-9e2e-5f8d1511ccb9 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:01.722010 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 685/1380] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:56:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:56:01.730727 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c10b0cc-e018-4c38-b97d-2dc10a6b44ee None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:01.731216 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c10b0cc-e018-4c38-b97d-2dc10a6b44ee None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:01.731619 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c10b0cc-e018-4c38-b97d-2dc10a6b44ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:01.757871 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-0c10b0cc-e018-4c38-b97d-2dc10a6b44ee None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:56:01.759385 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-0c10b0cc-e018-4c38-b97d-2dc10a6b44ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:56:01.772940 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 696/1381] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:56:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:56:01.824623 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ee1df-8f2f-478d-9e2e-5f8d1511ccb9 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:56:01.840151 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ee1df-8f2f-478d-9e2e-5f8d1511ccb9 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:56:01.845901 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b453e-d5b4-4d27-adea-f23b0309f753 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:56:01.846660 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b453e-d5b4-4d27-adea-f23b0309f753 service nova] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:01.847048 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b453e-d5b4-4d27-adea-f23b0309f753 service nova] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:01.847472 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b453e-d5b4-4d27-adea-f23b0309f753 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:01.857226 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4b453e-d5b4-4d27-adea-f23b0309f753 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:56:01.858907 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4b453e-d5b4-4d27-adea-f23b0309f753 service nova] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:56:01.868838 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 686/1382] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:56:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:56:04.190982 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09043321-efb3-45e5-8e28-67294e32633b None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:04.191607 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09043321-efb3-45e5-8e28-67294e32633b None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:04.191607 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09043321-efb3-45e5-8e28-67294e32633b None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:04.192957 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 697/1383] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:56:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:56:04.199696 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3771b-0e85-4a39-93ce-fcc6d6ac6611 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:04.200274 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3771b-0e85-4a39-93ce-fcc6d6ac6611 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:04.200274 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3771b-0e85-4a39-93ce-fcc6d6ac6611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:04.220494 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-71a3771b-0e85-4a39-93ce-fcc6d6ac6611 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:56:04.221757 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-71a3771b-0e85-4a39-93ce-fcc6d6ac6611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:56:04.234142 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 687/1384] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:56:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:56:04.276272 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09043321-efb3-45e5-8e28-67294e32633b None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:56:04.291536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09043321-efb3-45e5-8e28-67294e32633b None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:56:04.297106 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b81a788-65ca-4c53-96f0-e83f31de196f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:56:04.298008 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b81a788-65ca-4c53-96f0-e83f31de196f service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:04.298495 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b81a788-65ca-4c53-96f0-e83f31de196f service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:04.298958 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b81a788-65ca-4c53-96f0-e83f31de196f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:04.310519 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b81a788-65ca-4c53-96f0-e83f31de196f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:56:04.312587 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b81a788-65ca-4c53-96f0-e83f31de196f service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:56:04.324961 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 698/1385] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:56:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:56:06.839623 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21662b-5c48-415e-8970-6156122fedb5 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:06.840179 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21662b-5c48-415e-8970-6156122fedb5 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:06.840359 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21662b-5c48-415e-8970-6156122fedb5 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:06.842116 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 688/1386] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:56:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:56:06.849132 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814afc53-e39e-40aa-8550-00d9d9fb25f8 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:06.849540 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814afc53-e39e-40aa-8550-00d9d9fb25f8 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:06.849767 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814afc53-e39e-40aa-8550-00d9d9fb25f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:06.871882 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-814afc53-e39e-40aa-8550-00d9d9fb25f8 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:56:06.873443 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-814afc53-e39e-40aa-8550-00d9d9fb25f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:56:06.883792 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 699/1387] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:56:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:56:06.937708 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21662b-5c48-415e-8970-6156122fedb5 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:56:06.952805 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21662b-5c48-415e-8970-6156122fedb5 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:56:06.958314 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8e3619-ec27-4ae5-a019-ae51fd5969dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:56:06.958949 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e3619-ec27-4ae5-a019-ae51fd5969dc service nova] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:06.959230 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e3619-ec27-4ae5-a019-ae51fd5969dc service nova] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:06.960498 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e3619-ec27-4ae5-a019-ae51fd5969dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:06.968905 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8e3619-ec27-4ae5-a019-ae51fd5969dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:56:06.970473 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8e3619-ec27-4ae5-a019-ae51fd5969dc service nova] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:56:06.980561 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 689/1388] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:56:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:56:09.943742 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6972c169-b9f1-4d81-9531-90ff611b848f None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:09.944301 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6972c169-b9f1-4d81-9531-90ff611b848f None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:09.944515 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6972c169-b9f1-4d81-9531-90ff611b848f None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:09.946263 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 700/1389] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:56:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:56:09.952477 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad865c8-7989-4c33-b383-93974d959f9d None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:09.952871 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad865c8-7989-4c33-b383-93974d959f9d None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:09.953040 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad865c8-7989-4c33-b383-93974d959f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:09.971381 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-aad865c8-7989-4c33-b383-93974d959f9d None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:56:09.972687 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-aad865c8-7989-4c33-b383-93974d959f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:56:09.981619 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 690/1390] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:56:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:56:10.028686 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6972c169-b9f1-4d81-9531-90ff611b848f None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:56:10.044835 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6972c169-b9f1-4d81-9531-90ff611b848f None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:56:10.050871 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b6f0a-4997-473b-b6a5-c3eaa76ab535 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:56:10.051646 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b6f0a-4997-473b-b6a5-c3eaa76ab535 service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:10.051988 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b6f0a-4997-473b-b6a5-c3eaa76ab535 service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:10.052237 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b6f0a-4997-473b-b6a5-c3eaa76ab535 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:10.062308 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55b6f0a-4997-473b-b6a5-c3eaa76ab535 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:56:10.064088 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55b6f0a-4997-473b-b6a5-c3eaa76ab535 service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:56:10.074966 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 701/1391] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:56:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:56:12.611724 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d5cc9-8c1a-49b0-b2a6-3256e58a02a9 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:12.612370 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d5cc9-8c1a-49b0-b2a6-3256e58a02a9 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:12.612370 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d5cc9-8c1a-49b0-b2a6-3256e58a02a9 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:12.613667 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 691/1392] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:56:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:56:12.619919 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d79e7-e5bf-442c-9ff8-272d061b4546 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:12.620356 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d79e7-e5bf-442c-9ff8-272d061b4546 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:12.620606 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d79e7-e5bf-442c-9ff8-272d061b4546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:12.638066 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-a18d79e7-e5bf-442c-9ff8-272d061b4546 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:56:12.639248 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a18d79e7-e5bf-442c-9ff8-272d061b4546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:56:12.647486 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 702/1393] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:56:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:56:12.692232 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596d5cc9-8c1a-49b0-b2a6-3256e58a02a9 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:56:12.706551 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596d5cc9-8c1a-49b0-b2a6-3256e58a02a9 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:56:12.712192 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea39e9c-d03e-481f-8260-c33d566e42ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:56:12.712833 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea39e9c-d03e-481f-8260-c33d566e42ec service nova] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:12.713118 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea39e9c-d03e-481f-8260-c33d566e42ec service nova] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:12.713396 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea39e9c-d03e-481f-8260-c33d566e42ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:12.721361 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea39e9c-d03e-481f-8260-c33d566e42ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:56:12.722933 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea39e9c-d03e-481f-8260-c33d566e42ec service nova] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:56:12.732994 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 692/1394] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:56:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:56:13.512051 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3873b23-b50c-4c15-8f3a-73e0c5582d26 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:13.512335 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3873b23-b50c-4c15-8f3a-73e0c5582d26 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:13.512613 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3873b23-b50c-4c15-8f3a-73e0c5582d26 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:13.514008 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 703/1395] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:56:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:56:13.519364 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb1e75c-d882-4670-a6a5-4f574bd2fd15 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:13.519729 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb1e75c-d882-4670-a6a5-4f574bd2fd15 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:13.520019 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb1e75c-d882-4670-a6a5-4f574bd2fd15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:13.526742 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=52 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:56:13.539543 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-6cb1e75c-d882-4670-a6a5-4f574bd2fd15 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:56:13.540727 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-6cb1e75c-d882-4670-a6a5-4f574bd2fd15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:56:13.548464 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 693/1396] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:56:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:56:13.604919 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f02eb-2740-48db-b272-d1a1c5eec525 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:13.605215 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f02eb-2740-48db-b272-d1a1c5eec525 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:13.605496 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f02eb-2740-48db-b272-d1a1c5eec525 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:13.606865 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 704/1397] 10.222.0.13 () {54 vars in 962 bytes} [Tue Jun 7 21:56:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:56:13.611952 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840c637-3930-454c-a8b3-f424dc3d0d47 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:13.612426 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840c637-3930-454c-a8b3-f424dc3d0d47 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:13.612541 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840c637-3930-454c-a8b3-f424dc3d0d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:13.661046 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-6840c637-3930-454c-a8b3-f424dc3d0d47 None None] MFA Rules not processed for user `96987b04471944bc86ab44ff9936c732`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:56:13.662279 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-6840c637-3930-454c-a8b3-f424dc3d0d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:56:13.722412 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 694/1398] 10.222.0.13 () {58 vars in 1059 bytes} [Tue Jun 7 21:56:13 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:56:13.730398 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f02eb-2740-48db-b272-d1a1c5eec525 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:56:13.741378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f02eb-2740-48db-b272-d1a1c5eec525 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:56:13.744886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae9bf5-e23d-46b3-91d8-3fe38acdf1d2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96987b04471944bc86ab44ff9936c732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:56:13.745510 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae9bf5-e23d-46b3-91d8-3fe38acdf1d2 service glance] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:13.745795 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae9bf5-e23d-46b3-91d8-3fe38acdf1d2 service glance] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:13.746063 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae9bf5-e23d-46b3-91d8-3fe38acdf1d2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:13.755287 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ae9bf5-e23d-46b3-91d8-3fe38acdf1d2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:56:13.757635 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ae9bf5-e23d-46b3-91d8-3fe38acdf1d2 service glance] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:56:13.767952 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 705/1399] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:56:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:56:14.316400 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebbc5e8-e3d5-4667-bd48-353f7ca2772a None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:14.316777 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebbc5e8-e3d5-4667-bd48-353f7ca2772a None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:14.317131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebbc5e8-e3d5-4667-bd48-353f7ca2772a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:14.366856 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-5ebbc5e8-e3d5-4667-bd48-353f7ca2772a None None] MFA Rules not processed for user `86619a80df8841a3ba7d8471ac38056a`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:56:14.368096 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-5ebbc5e8-e3d5-4667-bd48-353f7ca2772a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:56:14.432398 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 695/1400] 10.222.0.13 () {58 vars in 1009 bytes} [Tue Jun 7 21:56:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:56:14.477012 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f283bc-5c6d-4dad-adef-2974268f0eff None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:14.477298 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f283bc-5c6d-4dad-adef-2974268f0eff None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:14.477600 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f283bc-5c6d-4dad-adef-2974268f0eff None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:14.478970 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 706/1401] 10.222.0.13 () {54 vars in 942 bytes} [Tue Jun 7 21:56:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 21:56:14.485312 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c95fd-277c-4864-a11b-cc9bfbc7e392 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:14.485601 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c95fd-277c-4864-a11b-cc9bfbc7e392 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:14.485911 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c95fd-277c-4864-a11b-cc9bfbc7e392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:14.534021 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-962c95fd-277c-4864-a11b-cc9bfbc7e392 None None] MFA Rules not processed for user `7a1261563b6d4e50a4ffeea6a32bdb01`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:56:14.535212 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-962c95fd-277c-4864-a11b-cc9bfbc7e392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:56:14.555360 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:56:14.614925 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 696/1402] 10.222.0.13 () {58 vars in 1039 bytes} [Tue Jun 7 21:56:14 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 21:56:14.624637 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f283bc-5c6d-4dad-adef-2974268f0eff None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:56:14.638781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f283bc-5c6d-4dad-adef-2974268f0eff None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:56:14.643756 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee5f464-e30f-4689-9cf0-b079e14054e6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1261563b6d4e50a4ffeea6a32bdb01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:56:14.644390 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5f464-e30f-4689-9cf0-b079e14054e6 service swift] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:14.644673 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5f464-e30f-4689-9cf0-b079e14054e6 service swift] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:14.644949 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5f464-e30f-4689-9cf0-b079e14054e6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:14.654013 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee5f464-e30f-4689-9cf0-b079e14054e6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:56:14.655652 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee5f464-e30f-4689-9cf0-b079e14054e6 service swift] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:56:14.664689 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 707/1403] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Jun 7 21:56:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 21:56:15.035976 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c4b2a-59ea-480b-8e1f-90248b15b8b0 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:15.036271 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c4b2a-59ea-480b-8e1f-90248b15b8b0 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:15.036558 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c4b2a-59ea-480b-8e1f-90248b15b8b0 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:15.038473 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 697/1404] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:56:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:56:15.043882 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2139820-56a0-4101-b6f8-1c9f7382b0ee None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:15.044684 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2139820-56a0-4101-b6f8-1c9f7382b0ee None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:15.044973 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2139820-56a0-4101-b6f8-1c9f7382b0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:15.064091 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-c2139820-56a0-4101-b6f8-1c9f7382b0ee None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:56:15.065601 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c2139820-56a0-4101-b6f8-1c9f7382b0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:56:15.074295 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 708/1405] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:56:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:56:15.120046 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01c4b2a-59ea-480b-8e1f-90248b15b8b0 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:56:15.139354 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01c4b2a-59ea-480b-8e1f-90248b15b8b0 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:56:15.144705 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ceb0c1-a4c3-4790-95dd-d04cd5733514 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:56:15.145345 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ceb0c1-a4c3-4790-95dd-d04cd5733514 service nova] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:15.145624 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ceb0c1-a4c3-4790-95dd-d04cd5733514 service nova] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:15.145919 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ceb0c1-a4c3-4790-95dd-d04cd5733514 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:15.154192 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ceb0c1-a4c3-4790-95dd-d04cd5733514 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:56:15.156993 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ceb0c1-a4c3-4790-95dd-d04cd5733514 service nova] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:56:15.166346 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 698/1406] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:56:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:56:17.546785 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e954a7-e485-47c5-963d-1699debad90b None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:17.547418 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e954a7-e485-47c5-963d-1699debad90b None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:17.547418 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e954a7-e485-47c5-963d-1699debad90b None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:17.548720 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 709/1407] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:56:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:56:17.554937 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8605b53a-0771-47f2-a5c4-2ab3de6f2e05 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:17.555399 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8605b53a-0771-47f2-a5c4-2ab3de6f2e05 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:17.555512 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8605b53a-0771-47f2-a5c4-2ab3de6f2e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:17.574037 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-8605b53a-0771-47f2-a5c4-2ab3de6f2e05 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:56:17.575532 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-8605b53a-0771-47f2-a5c4-2ab3de6f2e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:56:17.589420 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 699/1408] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:56:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:56:17.635693 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e954a7-e485-47c5-963d-1699debad90b None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:56:17.652058 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e954a7-e485-47c5-963d-1699debad90b None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:56:17.657914 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5111c444-a87c-4d41-9e98-15117ee04edb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:56:17.658545 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111c444-a87c-4d41-9e98-15117ee04edb service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:17.658839 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111c444-a87c-4d41-9e98-15117ee04edb service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:17.659098 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111c444-a87c-4d41-9e98-15117ee04edb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:17.671581 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5111c444-a87c-4d41-9e98-15117ee04edb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:56:17.673170 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5111c444-a87c-4d41-9e98-15117ee04edb service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:56:17.687956 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 710/1409] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:56:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:56:27.568484 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=33 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:56:27.683340 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=43 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:56:38.545946 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77137f91-0925-4421-84a3-0228d1c48f7c None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:38.546528 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77137f91-0925-4421-84a3-0228d1c48f7c None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:38.546528 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77137f91-0925-4421-84a3-0228d1c48f7c None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:38.547893 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 700/1410] 10.222.0.13 () {54 vars in 918 bytes} [Tue Jun 7 21:56:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:56:38.552988 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb9df9e-8dc9-4388-aba4-0c62fa127742 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:38.553433 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb9df9e-8dc9-4388-aba4-0c62fa127742 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:38.553575 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb9df9e-8dc9-4388-aba4-0c62fa127742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:38.571112 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-cdb9df9e-8dc9-4388-aba4-0c62fa127742 None None] MFA Rules not processed for user `02941c6ad2d54e48b1114d9e030c66c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:56:38.572343 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-cdb9df9e-8dc9-4388-aba4-0c62fa127742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:56:38.582794 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 711/1411] 10.222.0.13 () {58 vars in 1015 bytes} [Tue Jun 7 21:56:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:56:38.597928 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db8df7-b9d4-4d93-83ad-cb68b5689282 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:38.598288 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db8df7-b9d4-4d93-83ad-cb68b5689282 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:38.598915 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db8df7-b9d4-4d93-83ad-cb68b5689282 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:38.600998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 701/1412] 10.222.0.13 () {54 vars in 918 bytes} [Tue Jun 7 21:56:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:56:38.605707 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425287b9-3b33-4ba0-a412-c4618c3b6654 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:38.606004 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425287b9-3b33-4ba0-a412-c4618c3b6654 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:38.606287 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425287b9-3b33-4ba0-a412-c4618c3b6654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:38.624269 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-425287b9-3b33-4ba0-a412-c4618c3b6654 None None] MFA Rules not processed for user `dbe0303ea0764347bfb054870ebba41f`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:56:38.625585 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-425287b9-3b33-4ba0-a412-c4618c3b6654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:56:38.634695 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 712/1413] 10.222.0.13 () {58 vars in 1015 bytes} [Tue Jun 7 21:56:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:56:41.375117 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ddf98-6948-4782-a9e6-e6a19fe0ae14 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:41.375759 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ddf98-6948-4782-a9e6-e6a19fe0ae14 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:41.375759 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ddf98-6948-4782-a9e6-e6a19fe0ae14 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:41.377072 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 702/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 21:56:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:56:41.382373 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5931729-11ea-49f9-b364-3a83062b9eff None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:41.382779 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5931729-11ea-49f9-b364-3a83062b9eff None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:41.382928 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5931729-11ea-49f9-b364-3a83062b9eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:41.400996 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-c5931729-11ea-49f9-b364-3a83062b9eff None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:56:41.402174 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c5931729-11ea-49f9-b364-3a83062b9eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:56:41.409746 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 713/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 21:56:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 21:56:41.452887 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5ddf98-6948-4782-a9e6-e6a19fe0ae14 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:56:41.469905 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5ddf98-6948-4782-a9e6-e6a19fe0ae14 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:56:41.476895 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b9ff89-ef27-4200-9271-7fbb8494e901 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:56:41.477537 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9ff89-ef27-4200-9271-7fbb8494e901 service nova] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:41.477818 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9ff89-ef27-4200-9271-7fbb8494e901 service nova] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:41.478093 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9ff89-ef27-4200-9271-7fbb8494e901 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:41.486166 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b9ff89-ef27-4200-9271-7fbb8494e901 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:56:41.487767 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b9ff89-ef27-4200-9271-7fbb8494e901 service nova] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:56:41.497725 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 703/1416] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:56:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:56:42.486256 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76506e86-0b0b-49d2-b0c9-0d6440d9fcfb None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:42.486827 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76506e86-0b0b-49d2-b0c9-0d6440d9fcfb None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:42.486827 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76506e86-0b0b-49d2-b0c9-0d6440d9fcfb None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:42.488178 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 714/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 21:56:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:56:42.494002 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cefd1b-901b-430f-ae7f-f2c136d12bdb None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:42.494448 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cefd1b-901b-430f-ae7f-f2c136d12bdb None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:42.494544 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cefd1b-901b-430f-ae7f-f2c136d12bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:42.513390 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-42cefd1b-901b-430f-ae7f-f2c136d12bdb None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:56:42.514616 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-42cefd1b-901b-430f-ae7f-f2c136d12bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:56:42.522549 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 704/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 21:56:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:56:42.683570 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76506e86-0b0b-49d2-b0c9-0d6440d9fcfb None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:56:42.698179 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76506e86-0b0b-49d2-b0c9-0d6440d9fcfb None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:56:42.703444 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c65ceb-b197-4ae5-8a6c-4d55bcea5335 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:56:42.704069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c65ceb-b197-4ae5-8a6c-4d55bcea5335 service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:42.704345 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c65ceb-b197-4ae5-8a6c-4d55bcea5335 service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:42.704615 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c65ceb-b197-4ae5-8a6c-4d55bcea5335 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:42.713524 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c65ceb-b197-4ae5-8a6c-4d55bcea5335 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:56:42.715084 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c65ceb-b197-4ae5-8a6c-4d55bcea5335 service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:56:42.724915 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 715/1419] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:56:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:56:43.690249 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467cf1c6-2907-4689-823f-01985aa62ff2 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:43.690838 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467cf1c6-2907-4689-823f-01985aa62ff2 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:43.690838 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467cf1c6-2907-4689-823f-01985aa62ff2 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:43.692277 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 705/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 21:56:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:56:43.697894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc944e96-a88e-4f64-9bb2-0d8280d22291 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:43.698299 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc944e96-a88e-4f64-9bb2-0d8280d22291 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:43.698479 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc944e96-a88e-4f64-9bb2-0d8280d22291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:43.716915 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-fc944e96-a88e-4f64-9bb2-0d8280d22291 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:56:43.718115 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-fc944e96-a88e-4f64-9bb2-0d8280d22291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:56:43.726217 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 716/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 21:56:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:56:43.875865 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467cf1c6-2907-4689-823f-01985aa62ff2 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:56:43.890404 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467cf1c6-2907-4689-823f-01985aa62ff2 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:56:43.896137 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d453a97b-2376-4dce-9419-e028c9d9fb85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:56:43.897004 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453a97b-2376-4dce-9419-e028c9d9fb85 service nova] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:43.897298 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453a97b-2376-4dce-9419-e028c9d9fb85 service nova] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:43.897578 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453a97b-2376-4dce-9419-e028c9d9fb85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:43.905735 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d453a97b-2376-4dce-9419-e028c9d9fb85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:56:43.907305 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d453a97b-2376-4dce-9419-e028c9d9fb85 service nova] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:56:43.916784 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 706/1422] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:56:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:56:44.821599 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe67206-0d30-47ea-a429-948b920e28ce None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:44.822128 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe67206-0d30-47ea-a429-948b920e28ce None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:44.822128 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe67206-0d30-47ea-a429-948b920e28ce None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:44.823524 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 717/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 21:56:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:56:44.828471 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07266469-df17-4625-8c8a-4cf104fe9647 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:44.828923 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07266469-df17-4625-8c8a-4cf104fe9647 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:44.829021 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07266469-df17-4625-8c8a-4cf104fe9647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:44.847128 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-07266469-df17-4625-8c8a-4cf104fe9647 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:56:44.848370 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-07266469-df17-4625-8c8a-4cf104fe9647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:56:44.857104 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 707/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 21:56:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:56:45.006074 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe67206-0d30-47ea-a429-948b920e28ce None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:56:45.020666 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe67206-0d30-47ea-a429-948b920e28ce None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:56:45.026733 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5d3ef5-5046-434a-9776-4a604bfb7d6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:56:45.027391 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d3ef5-5046-434a-9776-4a604bfb7d6c service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:45.027674 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d3ef5-5046-434a-9776-4a604bfb7d6c service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:45.027945 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d3ef5-5046-434a-9776-4a604bfb7d6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:45.035869 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5d3ef5-5046-434a-9776-4a604bfb7d6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:56:45.037553 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5d3ef5-5046-434a-9776-4a604bfb7d6c service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:56:45.046749 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 718/1425] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:56:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:56:45.962961 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f48c5-dabe-429b-80c6-8a0b995f34ca None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:45.963579 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f48c5-dabe-429b-80c6-8a0b995f34ca None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:45.963579 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f48c5-dabe-429b-80c6-8a0b995f34ca None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:45.964870 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 708/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 21:56:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:56:45.969728 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeaebaa-bd89-4e22-9b5b-c53fdc5e7b3b None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:45.970128 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeaebaa-bd89-4e22-9b5b-c53fdc5e7b3b None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:45.970283 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeaebaa-bd89-4e22-9b5b-c53fdc5e7b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:45.988339 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-5aeaebaa-bd89-4e22-9b5b-c53fdc5e7b3b None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:56:45.990209 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-5aeaebaa-bd89-4e22-9b5b-c53fdc5e7b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:56:45.997752 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 719/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 21:56:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:56:46.147387 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f48c5-dabe-429b-80c6-8a0b995f34ca None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:56:46.161210 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f48c5-dabe-429b-80c6-8a0b995f34ca None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:56:46.167201 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101937d-d34e-40db-b217-4d8fdb29dd1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:56:46.167860 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101937d-d34e-40db-b217-4d8fdb29dd1f service nova] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:46.168140 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101937d-d34e-40db-b217-4d8fdb29dd1f service nova] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:46.168414 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101937d-d34e-40db-b217-4d8fdb29dd1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:46.176254 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a101937d-d34e-40db-b217-4d8fdb29dd1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:56:46.178194 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a101937d-d34e-40db-b217-4d8fdb29dd1f service nova] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:56:46.187297 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 709/1428] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:56:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:56:47.032775 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ce882-5b7f-422a-9265-16a973b087c7 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:47.033312 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ce882-5b7f-422a-9265-16a973b087c7 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:47.033312 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ce882-5b7f-422a-9265-16a973b087c7 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:47.034585 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 720/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 21:56:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:56:47.040065 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d359f-9da3-440a-94f6-2adeb40cf2ce None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:47.040534 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d359f-9da3-440a-94f6-2adeb40cf2ce None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:47.040645 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d359f-9da3-440a-94f6-2adeb40cf2ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:47.058552 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-fd8d359f-9da3-440a-94f6-2adeb40cf2ce None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:56:47.059784 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-fd8d359f-9da3-440a-94f6-2adeb40cf2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:56:47.067367 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 710/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 21:56:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:56:47.215142 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717ce882-5b7f-422a-9265-16a973b087c7 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:56:47.230254 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717ce882-5b7f-422a-9265-16a973b087c7 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:56:47.235616 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992d70e1-ffb7-4876-aa69-48ef67a2f57a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:56:47.236251 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d70e1-ffb7-4876-aa69-48ef67a2f57a service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:47.236536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d70e1-ffb7-4876-aa69-48ef67a2f57a service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:47.236814 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d70e1-ffb7-4876-aa69-48ef67a2f57a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:47.244854 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992d70e1-ffb7-4876-aa69-48ef67a2f57a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:56:47.246430 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992d70e1-ffb7-4876-aa69-48ef67a2f57a service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:56:47.255831 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 721/1431] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:56:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:56:48.107819 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd33d352-0648-4e24-a434-b706fff2ee9d None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:48.108385 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd33d352-0648-4e24-a434-b706fff2ee9d None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:48.108385 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd33d352-0648-4e24-a434-b706fff2ee9d None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:48.109757 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 711/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 21:56:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:56:48.115117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030fe67-b660-4692-b188-b145e3e6bf96 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:48.115708 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030fe67-b660-4692-b188-b145e3e6bf96 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:48.115708 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030fe67-b660-4692-b188-b145e3e6bf96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:48.134047 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-f030fe67-b660-4692-b188-b145e3e6bf96 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:56:48.135243 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-f030fe67-b660-4692-b188-b145e3e6bf96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:56:48.142894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 722/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 21:56:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:56:48.307518 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd33d352-0648-4e24-a434-b706fff2ee9d None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:56:48.322279 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd33d352-0648-4e24-a434-b706fff2ee9d None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:56:48.327864 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0151297e-e937-4a48-a40c-3c441fcbce82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:56:48.328825 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0151297e-e937-4a48-a40c-3c441fcbce82 service nova] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:48.329140 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0151297e-e937-4a48-a40c-3c441fcbce82 service nova] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:48.329448 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0151297e-e937-4a48-a40c-3c441fcbce82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:48.338603 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0151297e-e937-4a48-a40c-3c441fcbce82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:56:48.340359 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0151297e-e937-4a48-a40c-3c441fcbce82 service nova] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:56:48.350992 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 712/1434] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:56:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:56:49.210548 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a1777-9b4b-42da-b6c0-3236f34f1ffc None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:49.211215 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a1777-9b4b-42da-b6c0-3236f34f1ffc None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:49.211215 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a1777-9b4b-42da-b6c0-3236f34f1ffc None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:49.212516 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 723/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 21:56:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:56:49.217752 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd62b34-5c83-4ba5-8bef-24744c6797ed None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:49.218164 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd62b34-5c83-4ba5-8bef-24744c6797ed None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:49.218304 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd62b34-5c83-4ba5-8bef-24744c6797ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:49.236180 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-0cd62b34-5c83-4ba5-8bef-24744c6797ed None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:56:49.237392 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-0cd62b34-5c83-4ba5-8bef-24744c6797ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:56:49.245451 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 713/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 21:56:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:56:49.394221 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a1777-9b4b-42da-b6c0-3236f34f1ffc None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:56:49.408742 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a1777-9b4b-42da-b6c0-3236f34f1ffc None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:56:49.414142 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d2ed6c-0d76-4886-8bf1-bec9730b9ffe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:56:49.414769 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2ed6c-0d76-4886-8bf1-bec9730b9ffe service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:49.415050 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2ed6c-0d76-4886-8bf1-bec9730b9ffe service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:49.415341 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2ed6c-0d76-4886-8bf1-bec9730b9ffe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:49.423259 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d2ed6c-0d76-4886-8bf1-bec9730b9ffe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:56:49.424850 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d2ed6c-0d76-4886-8bf1-bec9730b9ffe service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:56:49.434531 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 724/1437] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:56:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:56:50.405027 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b6a55-477e-429e-94fc-c0fb5348f2bc None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:50.405550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b6a55-477e-429e-94fc-c0fb5348f2bc None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:50.405645 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b6a55-477e-429e-94fc-c0fb5348f2bc None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:50.406977 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 714/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 21:56:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:56:50.413194 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51309f-3ed7-4461-a9e8-edb0ccf656d2 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:50.413667 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51309f-3ed7-4461-a9e8-edb0ccf656d2 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:50.413779 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51309f-3ed7-4461-a9e8-edb0ccf656d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:50.439518 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-9b51309f-3ed7-4461-a9e8-edb0ccf656d2 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:56:50.440729 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-9b51309f-3ed7-4461-a9e8-edb0ccf656d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:56:50.450201 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 725/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 21:56:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 21:56:50.637582 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442b6a55-477e-429e-94fc-c0fb5348f2bc None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:56:50.652326 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442b6a55-477e-429e-94fc-c0fb5348f2bc None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:56:50.657896 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0e4905-d1c4-429e-84b2-f13bc1ea5ea9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:56:50.658531 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e4905-d1c4-429e-84b2-f13bc1ea5ea9 service nova] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:50.658811 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e4905-d1c4-429e-84b2-f13bc1ea5ea9 service nova] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:50.659084 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e4905-d1c4-429e-84b2-f13bc1ea5ea9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:50.667432 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0e4905-d1c4-429e-84b2-f13bc1ea5ea9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:56:50.669072 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0e4905-d1c4-429e-84b2-f13bc1ea5ea9 service nova] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:56:50.678368 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 715/1440] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:56:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:56:52.808141 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f399a70-b1ed-44f7-bf1b-03dc1c63b7ec None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:52.808663 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f399a70-b1ed-44f7-bf1b-03dc1c63b7ec None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:52.808763 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f399a70-b1ed-44f7-bf1b-03dc1c63b7ec None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:52.810049 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 726/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 21:56:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:56:52.815192 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc9b8af-badf-4d2a-b857-21bfcd310375 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:52.815616 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc9b8af-badf-4d2a-b857-21bfcd310375 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:52.815783 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc9b8af-badf-4d2a-b857-21bfcd310375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:52.834593 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-9cc9b8af-badf-4d2a-b857-21bfcd310375 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:56:52.835809 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-9cc9b8af-badf-4d2a-b857-21bfcd310375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:56:52.843390 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 716/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 21:56:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:56:52.881307 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f399a70-b1ed-44f7-bf1b-03dc1c63b7ec None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:56:52.891864 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f399a70-b1ed-44f7-bf1b-03dc1c63b7ec None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:56:52.895341 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cf004f-5a77-4488-9b52-a589cafa0af0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:56:52.896543 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cf004f-5a77-4488-9b52-a589cafa0af0 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:52.896798 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cf004f-5a77-4488-9b52-a589cafa0af0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:52.897055 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cf004f-5a77-4488-9b52-a589cafa0af0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:52.905743 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cf004f-5a77-4488-9b52-a589cafa0af0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:56:52.907162 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cf004f-5a77-4488-9b52-a589cafa0af0 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:56:52.916467 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 727/1443] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:56:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:56:55.350837 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bcefd9-9722-476a-8e5b-11853df3c2ff None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:55.351453 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bcefd9-9722-476a-8e5b-11853df3c2ff None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:55.351453 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bcefd9-9722-476a-8e5b-11853df3c2ff None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:55.353243 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 717/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 21:56:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:56:55.359459 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4e153-75da-4c67-bcbb-eb90c1c2deaa None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:55.359928 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4e153-75da-4c67-bcbb-eb90c1c2deaa None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:55.360032 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4e153-75da-4c67-bcbb-eb90c1c2deaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:55.378548 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-8dc4e153-75da-4c67-bcbb-eb90c1c2deaa None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:56:55.379780 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-8dc4e153-75da-4c67-bcbb-eb90c1c2deaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:56:55.388088 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 728/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 21:56:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:56:55.425151 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bcefd9-9722-476a-8e5b-11853df3c2ff None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:56:55.436193 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bcefd9-9722-476a-8e5b-11853df3c2ff None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:56:55.439640 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f81a29-c8fe-4da2-9c39-63ee7ba2642d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:56:55.440271 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f81a29-c8fe-4da2-9c39-63ee7ba2642d service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:55.440554 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f81a29-c8fe-4da2-9c39-63ee7ba2642d service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:55.440832 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f81a29-c8fe-4da2-9c39-63ee7ba2642d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:55.450295 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f81a29-c8fe-4da2-9c39-63ee7ba2642d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:56:55.451775 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f81a29-c8fe-4da2-9c39-63ee7ba2642d service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:56:55.460927 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 718/1446] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:56:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:56:58.025871 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9c1ca-3d56-4fdc-9177-296ccc6b3b99 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:58.026435 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9c1ca-3d56-4fdc-9177-296ccc6b3b99 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:58.026435 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9c1ca-3d56-4fdc-9177-296ccc6b3b99 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:58.027755 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 729/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 21:56:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:56:58.033076 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a87c8-b445-46f9-b215-18490cd9c9b9 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:58.033512 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a87c8-b445-46f9-b215-18490cd9c9b9 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:58.033646 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a87c8-b445-46f9-b215-18490cd9c9b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:58.051538 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-fc0a87c8-b445-46f9-b215-18490cd9c9b9 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:56:58.053524 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-fc0a87c8-b445-46f9-b215-18490cd9c9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:56:58.061401 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 719/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 21:56:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:56:58.099976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd9c1ca-3d56-4fdc-9177-296ccc6b3b99 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:56:58.110560 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd9c1ca-3d56-4fdc-9177-296ccc6b3b99 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:56:58.114045 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2317966c-cb68-4e1b-8677-21c2f9e6b2fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:56:58.115242 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317966c-cb68-4e1b-8677-21c2f9e6b2fa service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:56:58.115550 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317966c-cb68-4e1b-8677-21c2f9e6b2fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:56:58.115823 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317966c-cb68-4e1b-8677-21c2f9e6b2fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:56:58.124509 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2317966c-cb68-4e1b-8677-21c2f9e6b2fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:56:58.125932 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2317966c-cb68-4e1b-8677-21c2f9e6b2fa service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:56:58.135116 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 730/1449] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:56:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:57:00.534743 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27d607-312b-4504-9fc4-06d88b6cd09f None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:57:00.535319 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27d607-312b-4504-9fc4-06d88b6cd09f None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:57:00.535319 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27d607-312b-4504-9fc4-06d88b6cd09f None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:57:00.537089 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 720/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 21:57:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:57:00.542324 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb08c7d-4680-46b5-9f1e-b9cfe5e004cf None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:57:00.542767 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb08c7d-4680-46b5-9f1e-b9cfe5e004cf None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:57:00.542872 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb08c7d-4680-46b5-9f1e-b9cfe5e004cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:57:00.560531 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-edb08c7d-4680-46b5-9f1e-b9cfe5e004cf None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:57:00.561727 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-edb08c7d-4680-46b5-9f1e-b9cfe5e004cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:57:00.569967 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 731/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 21:57:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:57:00.608994 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc27d607-312b-4504-9fc4-06d88b6cd09f None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:57:00.610664 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-cc27d607-312b-4504-9fc4-06d88b6cd09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:57:00.624162 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc27d607-312b-4504-9fc4-06d88b6cd09f None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:57:00.627669 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e582ae-bf88-4403-add2-c2e31c2f53c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:57:00.628304 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e582ae-bf88-4403-add2-c2e31c2f53c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:57:00.628588 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e582ae-bf88-4403-add2-c2e31c2f53c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:57:00.628865 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e582ae-bf88-4403-add2-c2e31c2f53c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:57:00.638201 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e582ae-bf88-4403-add2-c2e31c2f53c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:57:00.639643 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e582ae-bf88-4403-add2-c2e31c2f53c6 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:57:00.649009 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 721/1452] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:57:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:57:03.056524 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ee170-4f71-4cbf-80ef-31fc0d19fc57 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:57:03.057112 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ee170-4f71-4cbf-80ef-31fc0d19fc57 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:57:03.057112 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ee170-4f71-4cbf-80ef-31fc0d19fc57 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:57:03.058436 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 732/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 21:57:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:57:03.063763 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b72e6-20e5-400d-aa59-b0bf37f804a3 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:57:03.064216 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b72e6-20e5-400d-aa59-b0bf37f804a3 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:57:03.064324 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b72e6-20e5-400d-aa59-b0bf37f804a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:57:03.082467 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-2b2b72e6-20e5-400d-aa59-b0bf37f804a3 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:57:03.084039 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-2b2b72e6-20e5-400d-aa59-b0bf37f804a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:57:03.091670 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 722/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 21:57:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 21:57:03.129156 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ee170-4f71-4cbf-80ef-31fc0d19fc57 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:57:03.140527 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ee170-4f71-4cbf-80ef-31fc0d19fc57 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:57:03.144070 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0595dba9-f63b-4919-a51f-d77fc810afca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:57:03.144702 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595dba9-f63b-4919-a51f-d77fc810afca service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:57:03.145024 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595dba9-f63b-4919-a51f-d77fc810afca service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:57:03.145364 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595dba9-f63b-4919-a51f-d77fc810afca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:57:03.154134 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0595dba9-f63b-4919-a51f-d77fc810afca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:57:03.155600 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0595dba9-f63b-4919-a51f-d77fc810afca service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:57:03.164869 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 733/1455] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:57:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:57:05.497645 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb6ed4-df11-48b1-8e01-1a3945fd328a None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:57:05.498239 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb6ed4-df11-48b1-8e01-1a3945fd328a None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:57:05.498239 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb6ed4-df11-48b1-8e01-1a3945fd328a None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:57:05.499644 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 723/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 21:57:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:57:05.505671 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d29f4f-34ef-40af-9853-a468ed1acc94 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:57:05.506112 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d29f4f-34ef-40af-9853-a468ed1acc94 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:57:05.506255 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d29f4f-34ef-40af-9853-a468ed1acc94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:57:05.524367 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-f5d29f4f-34ef-40af-9853-a468ed1acc94 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:57:05.525556 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-f5d29f4f-34ef-40af-9853-a468ed1acc94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:57:05.533192 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 734/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 21:57:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:57:05.570556 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb6ed4-df11-48b1-8e01-1a3945fd328a None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:57:05.581069 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb6ed4-df11-48b1-8e01-1a3945fd328a None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:57:05.585188 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a60bb0c-8c79-4850-9a54-ccafb501cfac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:57:05.585825 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a60bb0c-8c79-4850-9a54-ccafb501cfac service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:57:05.586107 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a60bb0c-8c79-4850-9a54-ccafb501cfac service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:57:05.586383 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a60bb0c-8c79-4850-9a54-ccafb501cfac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:57:05.595287 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a60bb0c-8c79-4850-9a54-ccafb501cfac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:57:05.596739 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a60bb0c-8c79-4850-9a54-ccafb501cfac service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:57:05.606394 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 724/1458] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:57:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:57:07.894310 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48224df7-2d8f-4128-bb3b-6cee2ad0e36d None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:57:07.894894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48224df7-2d8f-4128-bb3b-6cee2ad0e36d None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:57:07.894894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48224df7-2d8f-4128-bb3b-6cee2ad0e36d None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:57:07.896821 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 735/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 21:57:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:57:07.902079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a048fc-c924-4869-a7ad-b2f94d189798 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:57:07.902492 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a048fc-c924-4869-a7ad-b2f94d189798 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:57:07.902628 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a048fc-c924-4869-a7ad-b2f94d189798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:57:07.908781 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=70 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:57:07.921389 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-f5a048fc-c924-4869-a7ad-b2f94d189798 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:57:07.922587 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-f5a048fc-c924-4869-a7ad-b2f94d189798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:57:07.930311 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 725/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 21:57:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:57:07.969330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48224df7-2d8f-4128-bb3b-6cee2ad0e36d None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:57:07.975143 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=73 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:57:07.981499 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48224df7-2d8f-4128-bb3b-6cee2ad0e36d None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:57:07.985008 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78c890a-0937-41d0-9670-23495f6996ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:57:07.985644 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c890a-0937-41d0-9670-23495f6996ee service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:57:07.985929 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c890a-0937-41d0-9670-23495f6996ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:57:07.986208 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c890a-0937-41d0-9670-23495f6996ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:57:07.995305 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78c890a-0937-41d0-9670-23495f6996ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:57:07.996813 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78c890a-0937-41d0-9670-23495f6996ee service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:57:08.006693 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 736/1461] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:57:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:57:10.321301 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a489e88-60e6-4d6c-9f9f-596ad4dbbbda None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:57:10.321865 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a489e88-60e6-4d6c-9f9f-596ad4dbbbda None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:57:10.321865 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a489e88-60e6-4d6c-9f9f-596ad4dbbbda None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:57:10.323226 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 726/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 21:57:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:57:10.328480 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb869016-4366-408c-b0d0-df166e99332c None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:57:10.328926 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb869016-4366-408c-b0d0-df166e99332c None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:57:10.329062 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb869016-4366-408c-b0d0-df166e99332c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:57:10.346754 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-bb869016-4366-408c-b0d0-df166e99332c None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:57:10.347984 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-bb869016-4366-408c-b0d0-df166e99332c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:57:10.356554 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 737/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 21:57:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:57:10.393098 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a489e88-60e6-4d6c-9f9f-596ad4dbbbda None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:57:10.404386 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a489e88-60e6-4d6c-9f9f-596ad4dbbbda None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:57:10.407859 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed6186-2a75-4270-967e-1f2e1d2f689a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:57:10.408494 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed6186-2a75-4270-967e-1f2e1d2f689a service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:57:10.408775 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed6186-2a75-4270-967e-1f2e1d2f689a service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:57:10.409050 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed6186-2a75-4270-967e-1f2e1d2f689a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:57:10.418498 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeed6186-2a75-4270-967e-1f2e1d2f689a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:57:10.419972 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeed6186-2a75-4270-967e-1f2e1d2f689a service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:57:10.429068 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 727/1464] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:57:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:57:12.779596 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc532da-d9e0-4faf-96de-9b6a7b702b4f None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:57:12.780065 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc532da-d9e0-4faf-96de-9b6a7b702b4f None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:57:12.780188 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc532da-d9e0-4faf-96de-9b6a7b702b4f None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:57:12.781562 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 738/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 21:57:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:57:12.787468 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187ae76-be60-4142-b223-ff5b8eba71c7 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:57:12.787913 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187ae76-be60-4142-b223-ff5b8eba71c7 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:57:12.788021 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187ae76-be60-4142-b223-ff5b8eba71c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:57:12.805283 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-8187ae76-be60-4142-b223-ff5b8eba71c7 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:57:12.806481 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-8187ae76-be60-4142-b223-ff5b8eba71c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:57:12.814124 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 728/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 21:57:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:57:12.850749 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc532da-d9e0-4faf-96de-9b6a7b702b4f None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:57:12.861233 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc532da-d9e0-4faf-96de-9b6a7b702b4f None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:57:12.864813 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce107519-be3f-4c1b-b461-e13968610e8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:57:12.865432 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce107519-be3f-4c1b-b461-e13968610e8b service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:57:12.865741 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce107519-be3f-4c1b-b461-e13968610e8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:57:12.866006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce107519-be3f-4c1b-b461-e13968610e8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:57:12.874656 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce107519-be3f-4c1b-b461-e13968610e8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:57:12.876113 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce107519-be3f-4c1b-b461-e13968610e8b service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:57:12.885333 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 739/1467] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:57:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:57:15.163711 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a867878-6633-4187-adfc-90aed94fc435 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:57:15.164286 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a867878-6633-4187-adfc-90aed94fc435 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:57:15.164286 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a867878-6633-4187-adfc-90aed94fc435 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:57:15.165643 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 729/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 21:57:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:57:15.170959 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ccd3cc-060d-4331-8466-31161357d160 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:57:15.171375 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ccd3cc-060d-4331-8466-31161357d160 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:57:15.171557 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ccd3cc-060d-4331-8466-31161357d160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:57:15.189627 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-69ccd3cc-060d-4331-8466-31161357d160 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:57:15.190925 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-69ccd3cc-060d-4331-8466-31161357d160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:57:15.199398 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 740/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 21:57:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:57:15.237113 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a867878-6633-4187-adfc-90aed94fc435 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:57:15.247828 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a867878-6633-4187-adfc-90aed94fc435 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:57:15.255306 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ba710-7ae3-4c42-b4bc-d4016198c798 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:57:15.255969 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ba710-7ae3-4c42-b4bc-d4016198c798 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:57:15.256254 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ba710-7ae3-4c42-b4bc-d4016198c798 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:57:15.256534 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ba710-7ae3-4c42-b4bc-d4016198c798 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:57:15.265336 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100ba710-7ae3-4c42-b4bc-d4016198c798 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:57:15.266752 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100ba710-7ae3-4c42-b4bc-d4016198c798 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:57:15.276601 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 730/1470] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:57:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:57:25.184686 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=17 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:57:25.272484 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=17 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:57:47.248292 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4f553-42d3-41f4-85d7-f3960424c38e None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:57:47.248902 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4f553-42d3-41f4-85d7-f3960424c38e None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:57:47.248902 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4f553-42d3-41f4-85d7-f3960424c38e None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:57:47.250207 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 741/1471] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:57:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:57:47.255701 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f7fca-dacf-4631-b09c-0b8e416fee83 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:57:47.256154 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f7fca-dacf-4631-b09c-0b8e416fee83 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:57:47.256261 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f7fca-dacf-4631-b09c-0b8e416fee83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:57:47.274920 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-952f7fca-dacf-4631-b09c-0b8e416fee83 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:57:47.276153 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-952f7fca-dacf-4631-b09c-0b8e416fee83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:57:47.283912 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 731/1472] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:57:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 21:57:47.330775 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f17a2-3fec-4fa2-b1eb-130633efa7d0 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:57:47.331069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f17a2-3fec-4fa2-b1eb-130633efa7d0 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:57:47.331889 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f17a2-3fec-4fa2-b1eb-130633efa7d0 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:57:47.333311 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 742/1473] 10.222.0.13 () {54 vars in 962 bytes} [Tue Jun 7 21:57:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:57:47.338071 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e4a6a-f15e-4aa6-aca3-1b4b0d6ef509 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:57:47.338359 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e4a6a-f15e-4aa6-aca3-1b4b0d6ef509 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:57:47.338636 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e4a6a-f15e-4aa6-aca3-1b4b0d6ef509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:57:47.357948 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-8e5e4a6a-f15e-4aa6-aca3-1b4b0d6ef509 None None] MFA Rules not processed for user `96987b04471944bc86ab44ff9936c732`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:57:47.359121 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-8e5e4a6a-f15e-4aa6-aca3-1b4b0d6ef509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:57:47.366441 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 732/1474] 10.222.0.13 () {58 vars in 1059 bytes} [Tue Jun 7 21:57:47 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:57:47.373423 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068f17a2-3fec-4fa2-b1eb-130633efa7d0 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:57:47.384376 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068f17a2-3fec-4fa2-b1eb-130633efa7d0 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:57:47.387800 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc492a1-a4c8-4823-b76e-291ce145ba2e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96987b04471944bc86ab44ff9936c732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:57:47.388421 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc492a1-a4c8-4823-b76e-291ce145ba2e service glance] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:57:47.388699 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc492a1-a4c8-4823-b76e-291ce145ba2e service glance] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:57:47.388970 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc492a1-a4c8-4823-b76e-291ce145ba2e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:57:47.397734 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc492a1-a4c8-4823-b76e-291ce145ba2e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:57:47.399147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc492a1-a4c8-4823-b76e-291ce145ba2e service glance] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:57:47.408253 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 743/1475] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:57:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:57:49.783959 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae885c-6b45-4529-be2b-1ef949f5981d None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:57:49.785010 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae885c-6b45-4529-be2b-1ef949f5981d None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:57:49.785010 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae885c-6b45-4529-be2b-1ef949f5981d None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:57:49.789060 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 733/1476] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:57:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:57:49.794574 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a07dd-b75f-4af8-b718-44ddd2f228ef None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:57:49.795086 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a07dd-b75f-4af8-b718-44ddd2f228ef None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:57:49.795223 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a07dd-b75f-4af8-b718-44ddd2f228ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:57:49.813926 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-e85a07dd-b75f-4af8-b718-44ddd2f228ef None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:57:49.815129 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-e85a07dd-b75f-4af8-b718-44ddd2f228ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:57:49.826346 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 744/1477] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:57:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:57:49.855090 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae885c-6b45-4529-be2b-1ef949f5981d None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:57:49.869619 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae885c-6b45-4529-be2b-1ef949f5981d None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:57:49.874331 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03754e90-3f79-4016-9d43-8404e7fd62ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:57:49.875036 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03754e90-3f79-4016-9d43-8404e7fd62ac admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:57:49.875318 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03754e90-3f79-4016-9d43-8404e7fd62ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:57:49.876450 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03754e90-3f79-4016-9d43-8404e7fd62ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:57:49.876450 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03754e90-3f79-4016-9d43-8404e7fd62ac admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:57:49.878151 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: WARNING py.warnings [None req-03754e90-3f79-4016-9d43-8404e7fd62ac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 21:57:49.878151 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: warnings.warn(msg) Jun 07 21:57:49.878151 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: Jun 07 21:57:49.879408 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03754e90-3f79-4016-9d43-8404e7fd62ac admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:57:49.899272 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 734/1478] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jun 7 21:57:49 2022] GET /identity/v3/projects => generated 2553 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 21:57:52.276496 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68733af-2d3f-48e7-b1b8-69084a88edf5 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:57:52.277066 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68733af-2d3f-48e7-b1b8-69084a88edf5 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:57:52.277066 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68733af-2d3f-48e7-b1b8-69084a88edf5 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:57:52.278415 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 745/1479] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:57:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:57:52.283572 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26171e5c-37c2-4063-b7a0-a193b003854d None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:57:52.283973 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26171e5c-37c2-4063-b7a0-a193b003854d None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:57:52.284139 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26171e5c-37c2-4063-b7a0-a193b003854d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:57:52.301725 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-26171e5c-37c2-4063-b7a0-a193b003854d None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:57:52.302930 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-26171e5c-37c2-4063-b7a0-a193b003854d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:57:52.310884 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 735/1480] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:57:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:57:52.354331 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68733af-2d3f-48e7-b1b8-69084a88edf5 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:57:52.368850 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68733af-2d3f-48e7-b1b8-69084a88edf5 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:57:52.374136 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2b56a1-96b1-4c21-99ae-9268e7bcff98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:57:52.374771 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b56a1-96b1-4c21-99ae-9268e7bcff98 service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:57:52.375051 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b56a1-96b1-4c21-99ae-9268e7bcff98 service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:57:52.375358 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b56a1-96b1-4c21-99ae-9268e7bcff98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:57:52.383897 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2b56a1-96b1-4c21-99ae-9268e7bcff98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:57:52.385468 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2b56a1-96b1-4c21-99ae-9268e7bcff98 service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:57:52.394866 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 746/1481] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:57:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:57:54.827814 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9dcb9a-c560-43b9-8ff5-5d6b4d6e0d87 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:57:54.828363 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9dcb9a-c560-43b9-8ff5-5d6b4d6e0d87 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:57:54.828363 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9dcb9a-c560-43b9-8ff5-5d6b4d6e0d87 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:57:54.829746 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 736/1482] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:57:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:57:54.835245 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2301c5-c8a1-4a58-bb88-4dd2253699e2 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:57:54.835678 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2301c5-c8a1-4a58-bb88-4dd2253699e2 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:57:54.835901 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2301c5-c8a1-4a58-bb88-4dd2253699e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:57:54.853429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-2d2301c5-c8a1-4a58-bb88-4dd2253699e2 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:57:54.854616 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-2d2301c5-c8a1-4a58-bb88-4dd2253699e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:57:54.862268 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 747/1483] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:57:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:57:54.899036 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9dcb9a-c560-43b9-8ff5-5d6b4d6e0d87 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:57:54.909515 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9dcb9a-c560-43b9-8ff5-5d6b4d6e0d87 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:57:54.914223 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a4bcbd-0425-495e-8dd5-7e23f09dc007 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:57:54.914864 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4bcbd-0425-495e-8dd5-7e23f09dc007 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:57:54.915144 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4bcbd-0425-495e-8dd5-7e23f09dc007 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:57:54.915457 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4bcbd-0425-495e-8dd5-7e23f09dc007 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:57:54.925600 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a4bcbd-0425-495e-8dd5-7e23f09dc007 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:57:54.927170 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a4bcbd-0425-495e-8dd5-7e23f09dc007 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:57:54.939889 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 737/1484] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:57:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:57:57.265794 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd95bd2-e4ac-4615-84a3-4335fd9da211 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:57:57.266359 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd95bd2-e4ac-4615-84a3-4335fd9da211 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:57:57.266359 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd95bd2-e4ac-4615-84a3-4335fd9da211 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:57:57.267771 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 748/1485] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:57:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:57:57.273337 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f519f-b948-447d-899a-9733368653f0 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:57:57.273820 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f519f-b948-447d-899a-9733368653f0 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:57:57.273926 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f519f-b948-447d-899a-9733368653f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:57:57.292720 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-209f519f-b948-447d-899a-9733368653f0 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:57:57.293905 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-209f519f-b948-447d-899a-9733368653f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:57:57.302502 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 738/1486] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:57:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:57:57.341200 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd95bd2-e4ac-4615-84a3-4335fd9da211 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:57:57.352310 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd95bd2-e4ac-4615-84a3-4335fd9da211 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:57:57.355791 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb9eb4-8d65-4a83-98d8-270880e9493b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:57:57.356426 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb9eb4-8d65-4a83-98d8-270880e9493b service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:57:57.356713 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb9eb4-8d65-4a83-98d8-270880e9493b service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:57:57.356993 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb9eb4-8d65-4a83-98d8-270880e9493b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:57:57.365913 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdb9eb4-8d65-4a83-98d8-270880e9493b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:57:57.367351 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdb9eb4-8d65-4a83-98d8-270880e9493b service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:57:57.376615 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 749/1487] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:57:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:57:59.848244 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a215c59b-5833-4b95-9f15-c0814c535ec5 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:57:59.848742 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a215c59b-5833-4b95-9f15-c0814c535ec5 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:57:59.848847 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a215c59b-5833-4b95-9f15-c0814c535ec5 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:57:59.850433 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 739/1488] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:57:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:57:59.855650 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f6aee-2d2d-4a65-b4c0-26c01b7299b8 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:57:59.856059 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f6aee-2d2d-4a65-b4c0-26c01b7299b8 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:57:59.856209 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f6aee-2d2d-4a65-b4c0-26c01b7299b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:57:59.875192 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-1a0f6aee-2d2d-4a65-b4c0-26c01b7299b8 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:57:59.876410 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-1a0f6aee-2d2d-4a65-b4c0-26c01b7299b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:57:59.884173 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 750/1489] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:57:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:57:59.924529 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a215c59b-5833-4b95-9f15-c0814c535ec5 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:57:59.935023 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a215c59b-5833-4b95-9f15-c0814c535ec5 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:57:59.938909 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7606382-9d74-4e77-8baa-f816f782efb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:57:59.939564 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7606382-9d74-4e77-8baa-f816f782efb9 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:57:59.939847 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7606382-9d74-4e77-8baa-f816f782efb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:57:59.940125 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7606382-9d74-4e77-8baa-f816f782efb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:57:59.949312 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7606382-9d74-4e77-8baa-f816f782efb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:57:59.950733 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7606382-9d74-4e77-8baa-f816f782efb9 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:57:59.961139 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 740/1490] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:57:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:58:02.408404 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ec2de-c661-49f1-ba52-d72ccf07e957 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:58:02.408978 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ec2de-c661-49f1-ba52-d72ccf07e957 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:58:02.408978 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ec2de-c661-49f1-ba52-d72ccf07e957 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:58:02.411051 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 751/1491] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jun 7 21:58:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:58:02.416384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae8b09-8afb-483f-a692-27ace3c4ba63 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:58:02.416814 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae8b09-8afb-483f-a692-27ace3c4ba63 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:58:02.416930 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae8b09-8afb-483f-a692-27ace3c4ba63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:58:02.434656 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-ceae8b09-8afb-483f-a692-27ace3c4ba63 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:58:02.435885 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-ceae8b09-8afb-483f-a692-27ace3c4ba63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:58:02.444042 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 741/1492] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jun 7 21:58:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 21:58:02.483590 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ec2de-c661-49f1-ba52-d72ccf07e957 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:58:02.494166 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ec2de-c661-49f1-ba52-d72ccf07e957 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:58:02.497655 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b8ef96-56cd-4263-ad6f-6e7ba6a15cbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:58:02.498282 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b8ef96-56cd-4263-ad6f-6e7ba6a15cbb service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:58:02.498561 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b8ef96-56cd-4263-ad6f-6e7ba6a15cbb service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:58:02.498834 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b8ef96-56cd-4263-ad6f-6e7ba6a15cbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:58:02.508941 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b8ef96-56cd-4263-ad6f-6e7ba6a15cbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:58:02.510756 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b8ef96-56cd-4263-ad6f-6e7ba6a15cbb service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:58:02.522418 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 752/1493] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:58:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:58:12.429635 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=33 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:58:12.516852 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=33 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:59:28.681061 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c56126-bc91-44b3-a0ca-229761891018 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:59:28.681659 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c56126-bc91-44b3-a0ca-229761891018 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:59:28.681659 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c56126-bc91-44b3-a0ca-229761891018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:59:28.699718 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-68c56126-bc91-44b3-a0ca-229761891018 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:59:28.700907 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-68c56126-bc91-44b3-a0ca-229761891018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:59:28.708555 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 742/1494] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 21:59:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 21:59:28.716760 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeff01e-0038-4c82-ac0c-e2b01cb3572d None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:59:28.717283 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeff01e-0038-4c82-ac0c-e2b01cb3572d None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:59:28.717392 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeff01e-0038-4c82-ac0c-e2b01cb3572d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:59:28.736002 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-dbeff01e-0038-4c82-ac0c-e2b01cb3572d None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:59:28.737188 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-dbeff01e-0038-4c82-ac0c-e2b01cb3572d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:59:28.745004 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 753/1495] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 21:59:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:59:28.751441 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c56126-bc91-44b3-a0ca-229761891018 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:59:28.761177 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c56126-bc91-44b3-a0ca-229761891018 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:59:28.765673 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50b68aa-ae79-42d8-9bc6-17440dde2e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:59:28.766466 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b68aa-ae79-42d8-9bc6-17440dde2e53 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:59:28.766757 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b68aa-ae79-42d8-9bc6-17440dde2e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:59:28.767029 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b68aa-ae79-42d8-9bc6-17440dde2e53 admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:59:28.767782 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50b68aa-ae79-42d8-9bc6-17440dde2e53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 21:59:28.768087 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50b68aa-ae79-42d8-9bc6-17440dde2e53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:59:28.769097 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: WARNING py.warnings [None req-d50b68aa-ae79-42d8-9bc6-17440dde2e53 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 07 21:59:28.769097 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: warnings.warn(msg) Jun 07 21:59:28.769097 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: Jun 07 21:59:28.769977 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50b68aa-ae79-42d8-9bc6-17440dde2e53 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:59:28.781988 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 743/1496] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 21:59:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 21:59:28.790138 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeff01e-0038-4c82-ac0c-e2b01cb3572d None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:59:28.800523 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeff01e-0038-4c82-ac0c-e2b01cb3572d None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:59:28.805133 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319aeb25-f1ad-4063-b9d8-67e390019c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:59:28.805786 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319aeb25-f1ad-4063-b9d8-67e390019c9f admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:59:28.806071 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319aeb25-f1ad-4063-b9d8-67e390019c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:59:28.806347 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319aeb25-f1ad-4063-b9d8-67e390019c9f admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:59:28.807048 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319aeb25-f1ad-4063-b9d8-67e390019c9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:59:28.808817 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: WARNING py.warnings [None req-319aeb25-f1ad-4063-b9d8-67e390019c9f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 21:59:28.808817 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: warnings.warn(msg) Jun 07 21:59:28.808817 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: Jun 07 21:59:28.810012 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319aeb25-f1ad-4063-b9d8-67e390019c9f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:59:28.844509 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 754/1497] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 21:59:28 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 21:59:28.848836 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50b68aa-ae79-42d8-9bc6-17440dde2e53 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:59:28.858999 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50b68aa-ae79-42d8-9bc6-17440dde2e53 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:59:28.913941 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132a4528-8f5e-476e-9f0f-a419c19f1a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:59:28.914599 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a4528-8f5e-476e-9f0f-a419c19f1a99 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:59:28.914882 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a4528-8f5e-476e-9f0f-a419c19f1a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:59:28.915156 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a4528-8f5e-476e-9f0f-a419c19f1a99 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:59:28.915867 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132a4528-8f5e-476e-9f0f-a419c19f1a99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:59:28.916863 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: WARNING py.warnings [None req-132a4528-8f5e-476e-9f0f-a419c19f1a99 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 21:59:28.916863 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: warnings.warn(msg) Jun 07 21:59:28.916863 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: Jun 07 21:59:28.918046 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132a4528-8f5e-476e-9f0f-a419c19f1a99 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:59:28.937518 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 744/1498] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 21:59:28 2022] POST /identity/v3/users => generated 290 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 21:59:28.942162 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319aeb25-f1ad-4063-b9d8-67e390019c9f admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:59:28.952270 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319aeb25-f1ad-4063-b9d8-67e390019c9f admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:59:28.957357 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca5f9a-b9ee-4a58-aa7e-75e114d2bcb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:59:28.958010 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca5f9a-b9ee-4a58-aa7e-75e114d2bcb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:59:28.958292 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca5f9a-b9ee-4a58-aa7e-75e114d2bcb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:59:28.958563 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca5f9a-b9ee-4a58-aa7e-75e114d2bcb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:59:28.959283 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ca5f9a-b9ee-4a58-aa7e-75e114d2bcb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:59:28.960349 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: WARNING py.warnings [None req-63ca5f9a-b9ee-4a58-aa7e-75e114d2bcb6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 07 21:59:28.960349 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: warnings.warn(msg) Jun 07 21:59:28.960349 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: Jun 07 21:59:28.961218 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ca5f9a-b9ee-4a58-aa7e-75e114d2bcb6 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:59:28.973894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 755/1499] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 21:59:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 21:59:28.978179 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132a4528-8f5e-476e-9f0f-a419c19f1a99 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:59:28.987898 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132a4528-8f5e-476e-9f0f-a419c19f1a99 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:59:28.992620 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef90868a-7269-423a-b3a7-f81db33ec29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:59:28.993275 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90868a-7269-423a-b3a7-f81db33ec29e admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:59:28.993556 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90868a-7269-423a-b3a7-f81db33ec29e admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:59:28.993829 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90868a-7269-423a-b3a7-f81db33ec29e admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:59:28.994588 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef90868a-7269-423a-b3a7-f81db33ec29e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:59:28.995655 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: WARNING py.warnings [None req-ef90868a-7269-423a-b3a7-f81db33ec29e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 07 21:59:28.995655 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: warnings.warn(msg) Jun 07 21:59:28.995655 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: Jun 07 21:59:28.996537 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef90868a-7269-423a-b3a7-f81db33ec29e admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:59:29.010585 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 745/1500] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 21:59:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 21:59:29.014692 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca5f9a-b9ee-4a58-aa7e-75e114d2bcb6 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:59:29.025380 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca5f9a-b9ee-4a58-aa7e-75e114d2bcb6 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:59:29.029985 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a41f37-6695-40c8-8cf5-536cc8b0d1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:59:29.030870 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a41f37-6695-40c8-8cf5-536cc8b0d1e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:59:29.031157 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a41f37-6695-40c8-8cf5-536cc8b0d1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:59:29.031519 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a41f37-6695-40c8-8cf5-536cc8b0d1e3 admin admin] PATH_INFO: `/v3/projects/5ddf7518d5ec4f2ca0f5d83c39f20c47/users/5e1bb4bb94f84f17aaea5e1bb7e8df49/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:59:29.046810 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a41f37-6695-40c8-8cf5-536cc8b0d1e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ddf7518d5ec4f2ca0f5d83c39f20c47, user_id=5e1bb4bb94f84f17aaea5e1bb7e8df49, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:59:29.048037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: WARNING py.warnings [None req-67a41f37-6695-40c8-8cf5-536cc8b0d1e3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 21:59:29.048037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: warnings.warn(msg) Jun 07 21:59:29.048037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: Jun 07 21:59:29.049948 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a41f37-6695-40c8-8cf5-536cc8b0d1e3 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:59:29.062429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 756/1501] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 21:59:29 2022] PUT /identity/v3/projects/5ddf7518d5ec4f2ca0f5d83c39f20c47/users/5e1bb4bb94f84f17aaea5e1bb7e8df49/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 21:59:29.075737 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55bbd4-4d35-4621-bbb2-877f268ab94f None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:59:29.076031 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55bbd4-4d35-4621-bbb2-877f268ab94f None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:59:29.076311 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55bbd4-4d35-4621-bbb2-877f268ab94f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:59:29.103007 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-9d55bbd4-4d35-4621-bbb2-877f268ab94f None None] MFA Rules not processed for user `5e1bb4bb94f84f17aaea5e1bb7e8df49`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:59:29.104242 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-9d55bbd4-4d35-4621-bbb2-877f268ab94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:59:29.167689 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 746/1502] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 21:59:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:59:29.181374 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a41dc1-206d-4bc5-ab59-5eb2e64157b7 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:59:29.181665 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a41dc1-206d-4bc5-ab59-5eb2e64157b7 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:59:29.181944 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a41dc1-206d-4bc5-ab59-5eb2e64157b7 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:59:29.183302 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 757/1503] 10.222.0.13 () {54 vars in 963 bytes} [Tue Jun 7 21:59:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:59:29.188555 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b8fda-841d-4550-96a3-c4032c4c95cb None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:59:29.188778 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b8fda-841d-4550-96a3-c4032c4c95cb None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:59:29.189077 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b8fda-841d-4550-96a3-c4032c4c95cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:59:29.206463 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-be7b8fda-841d-4550-96a3-c4032c4c95cb None None] MFA Rules not processed for user `ea8a1e2cf1da4f87b8d4e16f525fc759`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:59:29.207681 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-be7b8fda-841d-4550-96a3-c4032c4c95cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:59:29.251747 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:59:29.260326 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 747/1504] 10.222.0.13 () {58 vars in 1060 bytes} [Tue Jun 7 21:59:29 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:59:29.268221 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a41dc1-206d-4bc5-ab59-5eb2e64157b7 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:59:29.278589 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a41dc1-206d-4bc5-ab59-5eb2e64157b7 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:59:29.282209 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a74fe62-5511-4508-b3de-14dd5b9c1ab2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:59:29.282839 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74fe62-5511-4508-b3de-14dd5b9c1ab2 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:59:29.283121 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74fe62-5511-4508-b3de-14dd5b9c1ab2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:59:29.283433 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74fe62-5511-4508-b3de-14dd5b9c1ab2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:59:29.291528 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a74fe62-5511-4508-b3de-14dd5b9c1ab2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:59:29.292990 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a74fe62-5511-4508-b3de-14dd5b9c1ab2 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:59:29.302147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 758/1505] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:59:29 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:59:29.343245 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5291471-2d62-4295-947f-5c58f8491ebf None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:59:29.343573 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5291471-2d62-4295-947f-5c58f8491ebf None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:59:29.343917 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5291471-2d62-4295-947f-5c58f8491ebf None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:59:29.345267 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 748/1506] 10.222.0.13 () {52 vars in 829 bytes} [Tue Jun 7 21:59:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 21:59:29.349173 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a74fe62-5511-4508-b3de-14dd5b9c1ab2 service cinder] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:59:29.427314 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a74fe62-5511-4508-b3de-14dd5b9c1ab2 service cinder] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:59:29.431901 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4950dcbb-12fd-42ff-8478-b0209cabad1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:59:29.432772 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950dcbb-12fd-42ff-8478-b0209cabad1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:59:29.433054 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950dcbb-12fd-42ff-8478-b0209cabad1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:59:29.433329 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950dcbb-12fd-42ff-8478-b0209cabad1f admin admin] PATH_INFO: `/v3/users/5e1bb4bb94f84f17aaea5e1bb7e8df49` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:59:29.434511 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4950dcbb-12fd-42ff-8478-b0209cabad1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e1bb4bb94f84f17aaea5e1bb7e8df49)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:59:29.435566 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: WARNING py.warnings [None req-4950dcbb-12fd-42ff-8478-b0209cabad1f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 21:59:29.435566 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: warnings.warn(msg) Jun 07 21:59:29.435566 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: Jun 07 21:59:29.436760 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4950dcbb-12fd-42ff-8478-b0209cabad1f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:59:29.500010 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-4950dcbb-12fd-42ff-8478-b0209cabad1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e1bb4bb94f84f17aaea5e1bb7e8df49'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 21:59:29.512668 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-4950dcbb-12fd-42ff-8478-b0209cabad1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e1bb4bb94f84f17aaea5e1bb7e8df49'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 21:59:29.532440 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-4950dcbb-12fd-42ff-8478-b0209cabad1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e1bb4bb94f84f17aaea5e1bb7e8df49'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 21:59:29.533137 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-4950dcbb-12fd-42ff-8478-b0209cabad1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e1bb4bb94f84f17aaea5e1bb7e8df49'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 21:59:29.533843 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-4950dcbb-12fd-42ff-8478-b0209cabad1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e1bb4bb94f84f17aaea5e1bb7e8df49'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 21:59:29.537717 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:59:29.550836 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 759/1507] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 21:59:29 2022] DELETE /identity/v3/users/5e1bb4bb94f84f17aaea5e1bb7e8df49 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 21:59:29.555239 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d5915-bd18-4a53-be25-da46f2d7aa4b None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:59:29.555550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d5915-bd18-4a53-be25-da46f2d7aa4b None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:59:29.555830 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d5915-bd18-4a53-be25-da46f2d7aa4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:59:29.574130 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-375d5915-bd18-4a53-be25-da46f2d7aa4b None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 21:59:29.575344 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-375d5915-bd18-4a53-be25-da46f2d7aa4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:59:29.633082 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 749/1508] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 21:59:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 21:59:29.642484 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4950dcbb-12fd-42ff-8478-b0209cabad1f admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:59:29.643914 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-4950dcbb-12fd-42ff-8478-b0209cabad1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:59:29.700799 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4950dcbb-12fd-42ff-8478-b0209cabad1f admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:59:29.704546 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e3eb65-7856-4ca5-a50f-9a2b47699915 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:59:29.705194 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3eb65-7856-4ca5-a50f-9a2b47699915 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:59:29.705477 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3eb65-7856-4ca5-a50f-9a2b47699915 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:59:29.705752 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3eb65-7856-4ca5-a50f-9a2b47699915 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:59:29.714552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e3eb65-7856-4ca5-a50f-9a2b47699915 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:59:29.716030 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e3eb65-7856-4ca5-a50f-9a2b47699915 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:59:29.726305 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 760/1509] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 21:59:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 21:59:30.074783 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375d5915-bd18-4a53-be25-da46f2d7aa4b None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 21:59:30.076265 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-375d5915-bd18-4a53-be25-da46f2d7aa4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 21:59:30.089523 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375d5915-bd18-4a53-be25-da46f2d7aa4b None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 21:59:30.095080 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7fd42-cfd5-4436-9b7d-39073ac7d90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 21:59:30.096001 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7fd42-cfd5-4436-9b7d-39073ac7d90a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 21:59:30.096283 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7fd42-cfd5-4436-9b7d-39073ac7d90a admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 21:59:30.096559 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7fd42-cfd5-4436-9b7d-39073ac7d90a admin admin] PATH_INFO: `/v3/projects/5ddf7518d5ec4f2ca0f5d83c39f20c47` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 21:59:30.099508 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e7fd42-cfd5-4436-9b7d-39073ac7d90a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ddf7518d5ec4f2ca0f5d83c39f20c47)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 21:59:30.100831 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: WARNING py.warnings [None req-14e7fd42-cfd5-4436-9b7d-39073ac7d90a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 21:59:30.100831 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: warnings.warn(msg) Jun 07 21:59:30.100831 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: Jun 07 21:59:30.102179 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e7fd42-cfd5-4436-9b7d-39073ac7d90a admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 21:59:30.167820 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-14e7fd42-cfd5-4436-9b7d-39073ac7d90a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ddf7518d5ec4f2ca0f5d83c39f20c47', 'request_id': 'req-14e7fd42-cfd5-4436-9b7d-39073ac7d90a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 21:59:30.177675 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-14e7fd42-cfd5-4436-9b7d-39073ac7d90a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ddf7518d5ec4f2ca0f5d83c39f20c47', 'request_id': 'req-14e7fd42-cfd5-4436-9b7d-39073ac7d90a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 21:59:30.190961 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-14e7fd42-cfd5-4436-9b7d-39073ac7d90a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ddf7518d5ec4f2ca0f5d83c39f20c47', 'request_id': 'req-14e7fd42-cfd5-4436-9b7d-39073ac7d90a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 21:59:30.201645 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-14e7fd42-cfd5-4436-9b7d-39073ac7d90a admin admin] The token cache is being invalidate because project 5ddf7518d5ec4f2ca0f5d83c39f20c47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 21:59:30.201928 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-14e7fd42-cfd5-4436-9b7d-39073ac7d90a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 21:59:30.202575 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-14e7fd42-cfd5-4436-9b7d-39073ac7d90a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 21:59:30.204851 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 750/1510] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 21:59:30 2022] DELETE /identity/v3/projects/5ddf7518d5ec4f2ca0f5d83c39f20c47 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 21:59:39.721952 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 21:59:40.198526 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:01:01.527951 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17246390-966a-4de0-886b-efbae90f24d9 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:01:01.528552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17246390-966a-4de0-886b-efbae90f24d9 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:01:01.528552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17246390-966a-4de0-886b-efbae90f24d9 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:01:01.529878 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 761/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 22:01:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 22:01:01.534879 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7145e5-f7b2-45a9-966c-a2df1e203494 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:01:01.535320 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7145e5-f7b2-45a9-966c-a2df1e203494 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:01:01.535489 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7145e5-f7b2-45a9-966c-a2df1e203494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:01:01.554161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-0b7145e5-f7b2-45a9-966c-a2df1e203494 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:01:01.555367 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-0b7145e5-f7b2-45a9-966c-a2df1e203494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:01:01.615680 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 751/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 22:01:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:01:01.785852 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17246390-966a-4de0-886b-efbae90f24d9 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:01:01.787592 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-17246390-966a-4de0-886b-efbae90f24d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:01:01.865205 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17246390-966a-4de0-886b-efbae90f24d9 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:01:01.870622 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8f69dd-e157-4bd6-abf2-a23e9fa62e01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:01:01.871310 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f69dd-e157-4bd6-abf2-a23e9fa62e01 service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:01:01.871626 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f69dd-e157-4bd6-abf2-a23e9fa62e01 service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:01:01.871904 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f69dd-e157-4bd6-abf2-a23e9fa62e01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:01:01.880623 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8f69dd-e157-4bd6-abf2-a23e9fa62e01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:01:01.882205 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8f69dd-e157-4bd6-abf2-a23e9fa62e01 service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:01:01.891677 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 762/1513] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:01:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:01:02.764732 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611d7a0-7cef-490f-b2b3-39568f04be5a None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:01:02.765194 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611d7a0-7cef-490f-b2b3-39568f04be5a None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:01:02.765374 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611d7a0-7cef-490f-b2b3-39568f04be5a None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:01:02.768010 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 752/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 22:01:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 22:01:02.772966 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437fcde-e685-49ae-8c63-7232864fdd0c None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:01:02.773415 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437fcde-e685-49ae-8c63-7232864fdd0c None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:01:02.773519 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437fcde-e685-49ae-8c63-7232864fdd0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:01:02.791425 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-0437fcde-e685-49ae-8c63-7232864fdd0c None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:01:02.792618 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-0437fcde-e685-49ae-8c63-7232864fdd0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:01:02.800999 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 763/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 22:01:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:01:02.952347 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f611d7a0-7cef-490f-b2b3-39568f04be5a None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:01:02.967255 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f611d7a0-7cef-490f-b2b3-39568f04be5a None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:01:02.973428 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a39390-4a40-4603-9e8e-3228a21f9ad9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:01:02.974071 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a39390-4a40-4603-9e8e-3228a21f9ad9 service nova] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:01:02.974357 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a39390-4a40-4603-9e8e-3228a21f9ad9 service nova] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:01:02.974633 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a39390-4a40-4603-9e8e-3228a21f9ad9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:01:02.982933 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a39390-4a40-4603-9e8e-3228a21f9ad9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:01:02.984604 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a39390-4a40-4603-9e8e-3228a21f9ad9 service nova] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:01:02.993998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 753/1516] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:01:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:01:03.247738 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d011046f-005d-4dda-8726-acc1009c820b None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:01:03.248024 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d011046f-005d-4dda-8726-acc1009c820b None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:01:03.248303 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d011046f-005d-4dda-8726-acc1009c820b None None] PATH_INFO: `/v3` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:01:03.249603 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 764/1517] 10.222.0.32 () {54 vars in 925 bytes} [Tue Jun 7 22:01:03 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:01:03.255753 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e198b32d-f83a-448c-a00d-5ce94cb26ca3 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:01:03.256046 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e198b32d-f83a-448c-a00d-5ce94cb26ca3 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:01:03.256331 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e198b32d-f83a-448c-a00d-5ce94cb26ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:01:03.274552 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-e198b32d-f83a-448c-a00d-5ce94cb26ca3 None None] MFA Rules not processed for user `02941c6ad2d54e48b1114d9e030c66c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:01:03.275784 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-e198b32d-f83a-448c-a00d-5ce94cb26ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:01:03.336644 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 754/1518] 10.222.0.32 () {58 vars in 1015 bytes} [Tue Jun 7 22:01:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:01:12.309840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53985b5f-8ac1-4cd9-886d-fc7af995879e None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:01:12.310444 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53985b5f-8ac1-4cd9-886d-fc7af995879e None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:01:12.310444 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53985b5f-8ac1-4cd9-886d-fc7af995879e None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:01:12.311771 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 765/1519] 10.222.0.13 () {54 vars in 960 bytes} [Tue Jun 7 22:01:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 22:01:12.317351 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8cfc5c-5bed-4924-ac73-fb4a1f0db2bc None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:01:12.317764 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8cfc5c-5bed-4924-ac73-fb4a1f0db2bc None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:01:12.317913 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8cfc5c-5bed-4924-ac73-fb4a1f0db2bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:01:12.336773 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-6a8cfc5c-5bed-4924-ac73-fb4a1f0db2bc None None] MFA Rules not processed for user `02941c6ad2d54e48b1114d9e030c66c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:01:12.337975 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-6a8cfc5c-5bed-4924-ac73-fb4a1f0db2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:01:12.347053 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 755/1520] 10.222.0.13 () {58 vars in 1057 bytes} [Tue Jun 7 22:01:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:01:12.354656 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53985b5f-8ac1-4cd9-886d-fc7af995879e None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:01:12.367923 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53985b5f-8ac1-4cd9-886d-fc7af995879e None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:01:12.372720 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c22ca7-7733-4d17-8752-4b40d41aa45a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02941c6ad2d54e48b1114d9e030c66c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:01:12.373345 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c22ca7-7733-4d17-8752-4b40d41aa45a service placement] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:01:12.373627 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c22ca7-7733-4d17-8752-4b40d41aa45a service placement] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:01:12.373900 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c22ca7-7733-4d17-8752-4b40d41aa45a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:01:12.382602 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c22ca7-7733-4d17-8752-4b40d41aa45a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:01:12.384202 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c22ca7-7733-4d17-8752-4b40d41aa45a service placement] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:01:12.394251 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 766/1521] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:01:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:01:14.301033 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f97063-8354-4357-bc6d-c30995c5b850 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:01:14.301561 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f97063-8354-4357-bc6d-c30995c5b850 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:01:14.301561 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f97063-8354-4357-bc6d-c30995c5b850 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:01:14.302975 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 756/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 22:01:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 22:01:14.307876 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeacad37-6c37-4925-b41b-5e77493f6175 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:01:14.308314 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeacad37-6c37-4925-b41b-5e77493f6175 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:01:14.308461 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeacad37-6c37-4925-b41b-5e77493f6175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:01:14.326550 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-aeacad37-6c37-4925-b41b-5e77493f6175 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:01:14.328285 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-aeacad37-6c37-4925-b41b-5e77493f6175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:01:14.336371 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 767/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 22:01:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:01:14.488898 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f97063-8354-4357-bc6d-c30995c5b850 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:01:14.503693 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f97063-8354-4357-bc6d-c30995c5b850 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:01:14.508965 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71409fb-289c-4c9d-b36b-001b8f1de65e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:01:14.509592 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71409fb-289c-4c9d-b36b-001b8f1de65e service nova] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:01:14.509873 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71409fb-289c-4c9d-b36b-001b8f1de65e service nova] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:01:14.510145 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71409fb-289c-4c9d-b36b-001b8f1de65e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:01:14.518687 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71409fb-289c-4c9d-b36b-001b8f1de65e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:01:14.520497 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71409fb-289c-4c9d-b36b-001b8f1de65e service nova] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:01:14.529851 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 757/1524] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:01:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:01:15.403917 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5d07f-ca8d-4fd1-806d-3fc0db3e29d5 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:01:15.404460 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5d07f-ca8d-4fd1-806d-3fc0db3e29d5 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:01:15.404460 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5d07f-ca8d-4fd1-806d-3fc0db3e29d5 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:01:15.406395 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 768/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 22:01:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 22:01:15.411929 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e1c7bb-e8e3-421f-aa9b-41c8c8c3fd78 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:01:15.412347 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e1c7bb-e8e3-421f-aa9b-41c8c8c3fd78 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:01:15.412514 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e1c7bb-e8e3-421f-aa9b-41c8c8c3fd78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:01:15.432391 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-61e1c7bb-e8e3-421f-aa9b-41c8c8c3fd78 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:01:15.433580 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-61e1c7bb-e8e3-421f-aa9b-41c8c8c3fd78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:01:15.441445 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 758/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 22:01:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:01:15.589908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e5d07f-ca8d-4fd1-806d-3fc0db3e29d5 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:01:15.605042 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e5d07f-ca8d-4fd1-806d-3fc0db3e29d5 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:01:15.610785 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c304a45-0b02-459b-86c2-b1f25a53e92d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:01:15.611446 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c304a45-0b02-459b-86c2-b1f25a53e92d service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:01:15.611731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c304a45-0b02-459b-86c2-b1f25a53e92d service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:01:15.612007 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c304a45-0b02-459b-86c2-b1f25a53e92d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:01:15.619961 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c304a45-0b02-459b-86c2-b1f25a53e92d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:01:15.621534 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c304a45-0b02-459b-86c2-b1f25a53e92d service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:01:15.630813 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 769/1527] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:01:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:01:19.630893 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c2a26-1597-41c9-8884-8c482adb58bc None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:01:19.631511 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c2a26-1597-41c9-8884-8c482adb58bc None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:01:19.631511 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c2a26-1597-41c9-8884-8c482adb58bc None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:01:19.632842 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 759/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 22:01:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 22:01:19.637964 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405898e-a21e-4ad6-adec-d32ebc030791 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:01:19.638417 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405898e-a21e-4ad6-adec-d32ebc030791 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:01:19.638530 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405898e-a21e-4ad6-adec-d32ebc030791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:01:19.657577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-e405898e-a21e-4ad6-adec-d32ebc030791 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:01:19.658773 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-e405898e-a21e-4ad6-adec-d32ebc030791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:01:19.666472 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 770/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 22:01:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:01:19.703867 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25c2a26-1597-41c9-8884-8c482adb58bc None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:01:19.705534 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-d25c2a26-1597-41c9-8884-8c482adb58bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:01:19.761818 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25c2a26-1597-41c9-8884-8c482adb58bc None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:01:19.765967 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc17e06-d841-4719-a879-5d3f05ac826a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:01:19.766614 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc17e06-d841-4719-a879-5d3f05ac826a service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:01:19.766899 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc17e06-d841-4719-a879-5d3f05ac826a service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:01:19.767178 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc17e06-d841-4719-a879-5d3f05ac826a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:01:19.776169 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc17e06-d841-4719-a879-5d3f05ac826a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:01:19.777591 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc17e06-d841-4719-a879-5d3f05ac826a service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:01:19.782821 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:01:19.788112 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 760/1530] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:01:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:01:22.137572 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895e054-cf6c-4023-805d-cb95c46c9347 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:01:22.138138 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895e054-cf6c-4023-805d-cb95c46c9347 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:01:22.138138 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895e054-cf6c-4023-805d-cb95c46c9347 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:01:22.139511 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 771/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 22:01:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 22:01:22.145299 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430df2fc-43a1-4271-8e06-1fbfdae92883 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:01:22.145703 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430df2fc-43a1-4271-8e06-1fbfdae92883 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:01:22.145861 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430df2fc-43a1-4271-8e06-1fbfdae92883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:01:22.163470 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-430df2fc-43a1-4271-8e06-1fbfdae92883 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:01:22.164667 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-430df2fc-43a1-4271-8e06-1fbfdae92883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:01:22.172360 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 761/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 22:01:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:01:22.207757 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8895e054-cf6c-4023-805d-cb95c46c9347 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:01:22.218419 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8895e054-cf6c-4023-805d-cb95c46c9347 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:01:22.222065 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cf945c-9b0f-40f2-b67e-d5493e8319aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:01:22.222690 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf945c-9b0f-40f2-b67e-d5493e8319aa service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:01:22.222971 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf945c-9b0f-40f2-b67e-d5493e8319aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:01:22.223241 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf945c-9b0f-40f2-b67e-d5493e8319aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:01:22.232246 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cf945c-9b0f-40f2-b67e-d5493e8319aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:01:22.233678 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cf945c-9b0f-40f2-b67e-d5493e8319aa service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:01:22.243121 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 772/1533] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:01:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:01:25.986931 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e602d-e2c7-4314-a8c7-70589bed2f89 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:01:25.987532 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e602d-e2c7-4314-a8c7-70589bed2f89 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:01:25.987532 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e602d-e2c7-4314-a8c7-70589bed2f89 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:01:25.988896 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 762/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 22:01:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 22:01:25.994202 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d22b7b2-d575-47a9-8051-e5430c96b3ca None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:01:25.994648 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d22b7b2-d575-47a9-8051-e5430c96b3ca None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:01:25.994784 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d22b7b2-d575-47a9-8051-e5430c96b3ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:01:26.013444 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-4d22b7b2-d575-47a9-8051-e5430c96b3ca None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:01:26.014578 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-4d22b7b2-d575-47a9-8051-e5430c96b3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:01:26.022964 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 773/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 22:01:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:01:26.293640 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e602d-e2c7-4314-a8c7-70589bed2f89 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:01:26.295300 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-095e602d-e2c7-4314-a8c7-70589bed2f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:01:26.352340 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e602d-e2c7-4314-a8c7-70589bed2f89 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:01:26.355876 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ae33a-a92f-4bc1-a8da-0399b0b355fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:01:26.356512 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ae33a-a92f-4bc1-a8da-0399b0b355fc service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:01:26.356795 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ae33a-a92f-4bc1-a8da-0399b0b355fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:01:26.357071 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ae33a-a92f-4bc1-a8da-0399b0b355fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:01:26.366428 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ae33a-a92f-4bc1-a8da-0399b0b355fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:01:26.367882 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ae33a-a92f-4bc1-a8da-0399b0b355fc service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:01:26.377452 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 763/1536] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:01:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:01:28.621531 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a669eda3-8ec9-4113-8432-ed188c407dbb None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:01:28.622406 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a669eda3-8ec9-4113-8432-ed188c407dbb None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:01:28.622683 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a669eda3-8ec9-4113-8432-ed188c407dbb None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:01:28.624766 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 774/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 22:01:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 22:01:28.630389 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6096c620-2bc6-42e5-a708-ea885050e66e None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:01:28.630896 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6096c620-2bc6-42e5-a708-ea885050e66e None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:01:28.631025 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6096c620-2bc6-42e5-a708-ea885050e66e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:01:28.648787 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-6096c620-2bc6-42e5-a708-ea885050e66e None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:01:28.649995 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-6096c620-2bc6-42e5-a708-ea885050e66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:01:28.658337 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 764/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 22:01:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:01:28.932843 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a669eda3-8ec9-4113-8432-ed188c407dbb None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:01:28.934575 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a669eda3-8ec9-4113-8432-ed188c407dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:01:28.948241 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a669eda3-8ec9-4113-8432-ed188c407dbb None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:01:28.951905 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b0128c-5cb9-4acd-bc1f-a398b6ee2618 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:01:28.952838 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b0128c-5cb9-4acd-bc1f-a398b6ee2618 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:01:28.953131 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b0128c-5cb9-4acd-bc1f-a398b6ee2618 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:01:28.953408 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b0128c-5cb9-4acd-bc1f-a398b6ee2618 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:01:28.964135 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b0128c-5cb9-4acd-bc1f-a398b6ee2618 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:01:28.965645 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b0128c-5cb9-4acd-bc1f-a398b6ee2618 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:01:28.977910 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 775/1539] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:01:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:01:38.643489 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=32 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:01:38.973210 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=70 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:02:59.569111 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c7cf6-23f2-4147-8164-3e964bca2d92 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:02:59.570381 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c7cf6-23f2-4147-8164-3e964bca2d92 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:02:59.570913 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c7cf6-23f2-4147-8164-3e964bca2d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:02:59.609016 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-9f0c7cf6-23f2-4147-8164-3e964bca2d92 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:02:59.610347 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-9f0c7cf6-23f2-4147-8164-3e964bca2d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:02:59.623386 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 765/1540] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:02:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:02:59.634276 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e9e5da-fe70-4d90-95dc-4dfe19b0b8e7 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:02:59.634869 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e9e5da-fe70-4d90-95dc-4dfe19b0b8e7 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:02:59.634869 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e9e5da-fe70-4d90-95dc-4dfe19b0b8e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:02:59.661441 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-25e9e5da-fe70-4d90-95dc-4dfe19b0b8e7 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:02:59.663055 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-25e9e5da-fe70-4d90-95dc-4dfe19b0b8e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:02:59.674815 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 776/1541] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:02:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:02:59.679816 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0c7cf6-23f2-4147-8164-3e964bca2d92 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:02:59.698326 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0c7cf6-23f2-4147-8164-3e964bca2d92 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:02:59.706113 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26db460-7d68-4d03-bc68-f7b56bd46a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:02:59.706796 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26db460-7d68-4d03-bc68-f7b56bd46a8c admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:02:59.707082 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26db460-7d68-4d03-bc68-f7b56bd46a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:02:59.707384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26db460-7d68-4d03-bc68-f7b56bd46a8c admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:02:59.708129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26db460-7d68-4d03-bc68-f7b56bd46a8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:02:59.708439 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26db460-7d68-4d03-bc68-f7b56bd46a8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:02:59.710061 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26db460-7d68-4d03-bc68-f7b56bd46a8c admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:02:59.729890 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 766/1542] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:02:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:02:59.739678 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e9e5da-fe70-4d90-95dc-4dfe19b0b8e7 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:02:59.750149 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e9e5da-fe70-4d90-95dc-4dfe19b0b8e7 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:02:59.760282 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7578fac-fc79-423f-bdf5-296889719606 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:02:59.760580 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7578fac-fc79-423f-bdf5-296889719606 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:02:59.760862 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7578fac-fc79-423f-bdf5-296889719606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:02:59.762698 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d4d06c-8c9e-4864-8948-15c4c571dd25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:02:59.763713 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4d06c-8c9e-4864-8948-15c4c571dd25 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:02:59.764333 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4d06c-8c9e-4864-8948-15c4c571dd25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:02:59.764834 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4d06c-8c9e-4864-8948-15c4c571dd25 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:02:59.766223 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d4d06c-8c9e-4864-8948-15c4c571dd25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:02:59.768427 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d4d06c-8c9e-4864-8948-15c4c571dd25 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:02:59.798798 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-e7578fac-fc79-423f-bdf5-296889719606 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:02:59.800209 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-e7578fac-fc79-423f-bdf5-296889719606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:02:59.813097 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 767/1543] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:02:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:02:59.816359 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 777/1544] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:02:59 2022] POST /identity/v3/projects => generated 363 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:02:59.820128 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7578fac-fc79-423f-bdf5-296889719606 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:02:59.822439 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c364f2d3-268e-48d1-b0ea-5a67bdedbb1c None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:02:59.823027 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c364f2d3-268e-48d1-b0ea-5a67bdedbb1c None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:02:59.823558 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c364f2d3-268e-48d1-b0ea-5a67bdedbb1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:02:59.830715 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7578fac-fc79-423f-bdf5-296889719606 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:02:59.863355 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-c364f2d3-268e-48d1-b0ea-5a67bdedbb1c None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:02:59.864589 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c364f2d3-268e-48d1-b0ea-5a67bdedbb1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:02:59.965551 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60891d80-a14f-4e98-8a5a-6f4fedef0634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:02:59.966264 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60891d80-a14f-4e98-8a5a-6f4fedef0634 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:02:59.966550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60891d80-a14f-4e98-8a5a-6f4fedef0634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:02:59.966829 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60891d80-a14f-4e98-8a5a-6f4fedef0634 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:02:59.967886 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60891d80-a14f-4e98-8a5a-6f4fedef0634 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:02:59.970033 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60891d80-a14f-4e98-8a5a-6f4fedef0634 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:02:59.990356 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 778/1545] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:02:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:02:59.993211 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc472908-c751-422b-a564-66b85799703b None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:02:59.993456 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc472908-c751-422b-a564-66b85799703b None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:02:59.993804 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc472908-c751-422b-a564-66b85799703b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:00.004062 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 768/1546] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:02:59 2022] POST /identity/v3/users => generated 282 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:03:00.005767 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b45e8-6bb8-4dce-bc63-ab54bd59de0c None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:00.005912 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b45e8-6bb8-4dce-bc63-ab54bd59de0c None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:00.006148 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b45e8-6bb8-4dce-bc63-ab54bd59de0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:00.015541 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-bc472908-c751-422b-a564-66b85799703b None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:00.017216 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-bc472908-c751-422b-a564-66b85799703b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:00.026592 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-296b45e8-6bb8-4dce-bc63-ab54bd59de0c None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:00.026815 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 779/1547] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:02:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:00.027979 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-296b45e8-6bb8-4dce-bc63-ab54bd59de0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:00.028452 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c620d2df-0cd8-4bb7-b0aa-3e59cccd7c52 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:00.028737 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c620d2df-0cd8-4bb7-b0aa-3e59cccd7c52 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:00.029012 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c620d2df-0cd8-4bb7-b0aa-3e59cccd7c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:00.043896 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 769/1548] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:00.045830 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca5fd7-26ed-4592-a5d3-4a0ae7cbead6 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:00.046171 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca5fd7-26ed-4592-a5d3-4a0ae7cbead6 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:00.046502 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca5fd7-26ed-4592-a5d3-4a0ae7cbead6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:00.053561 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-c620d2df-0cd8-4bb7-b0aa-3e59cccd7c52 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:00.054780 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c620d2df-0cd8-4bb7-b0aa-3e59cccd7c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:00.063899 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 780/1549] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:00.065350 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eaa7d5-4262-4196-817e-fbcec5fb4ed9 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:00.065482 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-dbca5fd7-26ed-4592-a5d3-4a0ae7cbead6 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:00.065666 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eaa7d5-4262-4196-817e-fbcec5fb4ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:00.065922 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eaa7d5-4262-4196-817e-fbcec5fb4ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:00.066543 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-dbca5fd7-26ed-4592-a5d3-4a0ae7cbead6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:00.074457 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 770/1550] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:00.075275 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbca5fd7-26ed-4592-a5d3-4a0ae7cbead6 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:00.084600 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbca5fd7-26ed-4592-a5d3-4a0ae7cbead6 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:00.084885 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-68eaa7d5-4262-4196-817e-fbcec5fb4ed9 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:00.086049 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-68eaa7d5-4262-4196-817e-fbcec5fb4ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:00.089156 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae0c26a-459f-4d5b-8a8b-f2ac6af81429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:00.089966 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae0c26a-459f-4d5b-8a8b-f2ac6af81429 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:00.090467 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae0c26a-459f-4d5b-8a8b-f2ac6af81429 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:00.090702 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae0c26a-459f-4d5b-8a8b-f2ac6af81429 admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:00.091463 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae0c26a-459f-4d5b-8a8b-f2ac6af81429 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:03:00.091777 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae0c26a-459f-4d5b-8a8b-f2ac6af81429 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:00.093375 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae0c26a-459f-4d5b-8a8b-f2ac6af81429 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:00.093941 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 781/1551] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:00.095168 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e15f5-dbca-43ed-90d9-850d01fc7e04 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:00.095498 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e15f5-dbca-43ed-90d9-850d01fc7e04 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:00.095762 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e15f5-dbca-43ed-90d9-850d01fc7e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:00.106890 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 771/1552] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:03:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:03:00.108743 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae0c26a-459f-4d5b-8a8b-f2ac6af81429 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:00.114795 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-151e15f5-dbca-43ed-90d9-850d01fc7e04 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:00.115855 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-151e15f5-dbca-43ed-90d9-850d01fc7e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:00.119536 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae0c26a-459f-4d5b-8a8b-f2ac6af81429 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:00.124404 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 782/1553] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:00.124643 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc30e5-9755-46f8-ab65-b27cfda87f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:00.125278 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc30e5-9755-46f8-ab65-b27cfda87f7a admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:00.125567 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc30e5-9755-46f8-ab65-b27cfda87f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:00.125676 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b279a-b1fe-4d6e-a662-5f4594140b5b None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:00.125864 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc30e5-9755-46f8-ab65-b27cfda87f7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:00.125966 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b279a-b1fe-4d6e-a662-5f4594140b5b None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:00.126180 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b279a-b1fe-4d6e-a662-5f4594140b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:00.126631 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bc30e5-9755-46f8-ab65-b27cfda87f7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:00.128267 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bc30e5-9755-46f8-ab65-b27cfda87f7a admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:00.139461 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 772/1554] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:00.140539 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadc3839-1e40-4bc9-9b4f-3fa2dcbad447 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:00.140827 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadc3839-1e40-4bc9-9b4f-3fa2dcbad447 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:00.141108 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadc3839-1e40-4bc9-9b4f-3fa2dcbad447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:00.147457 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-dc2b279a-b1fe-4d6e-a662-5f4594140b5b None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:00.148665 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-dc2b279a-b1fe-4d6e-a662-5f4594140b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:00.157293 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 783/1555] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:00.158338 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8377ae88-ab33-4094-85a6-1e5855b8bbbb None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:00.158623 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8377ae88-ab33-4094-85a6-1e5855b8bbbb None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:00.158899 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8377ae88-ab33-4094-85a6-1e5855b8bbbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:00.159011 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-eadc3839-1e40-4bc9-9b4f-3fa2dcbad447 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:00.160177 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-eadc3839-1e40-4bc9-9b4f-3fa2dcbad447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:00.169520 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 773/1556] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:00.170762 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2b2c6-cdcc-413b-98ac-8f73f6fb675e None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:00.171052 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2b2c6-cdcc-413b-98ac-8f73f6fb675e None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:00.171364 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2b2c6-cdcc-413b-98ac-8f73f6fb675e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:00.177487 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-8377ae88-ab33-4094-85a6-1e5855b8bbbb None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:00.180658 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-8377ae88-ab33-4094-85a6-1e5855b8bbbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:00.190630 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 784/1557] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:00.192004 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958eb24-d6e3-41c1-ad54-91950c464a73 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:00.192294 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958eb24-d6e3-41c1-ad54-91950c464a73 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:00.192570 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958eb24-d6e3-41c1-ad54-91950c464a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:00.197091 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-a0c2b2c6-cdcc-413b-98ac-8f73f6fb675e None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:00.198297 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-a0c2b2c6-cdcc-413b-98ac-8f73f6fb675e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:00.206100 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 774/1558] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:00.206870 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c2b2c6-cdcc-413b-98ac-8f73f6fb675e None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:00.210812 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-9958eb24-d6e3-41c1-ad54-91950c464a73 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:00.212029 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-9958eb24-d6e3-41c1-ad54-91950c464a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:00.216824 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c2b2c6-cdcc-413b-98ac-8f73f6fb675e None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:00.220513 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 785/1559] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:00.221528 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eb3c4c-5109-4c30-8fc2-bc91fc73dddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:00.221731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb942d25-de9a-4698-9491-b78f1e820e7b None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:00.222018 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb942d25-de9a-4698-9491-b78f1e820e7b None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:00.222177 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb3c4c-5109-4c30-8fc2-bc91fc73dddb admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:00.222298 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb942d25-de9a-4698-9491-b78f1e820e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:00.222461 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb3c4c-5109-4c30-8fc2-bc91fc73dddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:00.222738 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb3c4c-5109-4c30-8fc2-bc91fc73dddb admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:00.223514 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eb3c4c-5109-4c30-8fc2-bc91fc73dddb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:00.224476 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: WARNING py.warnings [None req-96eb3c4c-5109-4c30-8fc2-bc91fc73dddb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 22:03:00.224476 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: warnings.warn(msg) Jun 07 22:03:00.224476 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: Jun 07 22:03:00.225903 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eb3c4c-5109-4c30-8fc2-bc91fc73dddb admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:00.241005 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-cb942d25-de9a-4698-9491-b78f1e820e7b None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:00.242311 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-cb942d25-de9a-4698-9491-b78f1e820e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:00.251173 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 786/1560] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:00.251999 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb942d25-de9a-4698-9491-b78f1e820e7b None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:00.258449 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 775/1561] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:03:00 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:00.258449 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eb3c4c-5109-4c30-8fc2-bc91fc73dddb admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:00.261248 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb942d25-de9a-4698-9491-b78f1e820e7b None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:00.312179 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:00.319549 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095fb459-c560-47cd-ab3b-a3d288b7dc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:00.320293 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095fb459-c560-47cd-ab3b-a3d288b7dc39 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:00.320745 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095fb459-c560-47cd-ab3b-a3d288b7dc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:00.321033 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095fb459-c560-47cd-ab3b-a3d288b7dc39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:00.321791 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095fb459-c560-47cd-ab3b-a3d288b7dc39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:00.323475 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095fb459-c560-47cd-ab3b-a3d288b7dc39 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:00.324849 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eb3c4c-5109-4c30-8fc2-bc91fc73dddb admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:00.329383 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783c1ee-3b83-4cc5-aecc-29cd3cd152c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:00.330042 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783c1ee-3b83-4cc5-aecc-29cd3cd152c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:00.330332 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783c1ee-3b83-4cc5-aecc-29cd3cd152c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:00.330607 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783c1ee-3b83-4cc5-aecc-29cd3cd152c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:00.331358 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2783c1ee-3b83-4cc5-aecc-29cd3cd152c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:03:00.331659 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2783c1ee-3b83-4cc5-aecc-29cd3cd152c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:00.333235 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2783c1ee-3b83-4cc5-aecc-29cd3cd152c9 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:00.333738 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 787/1562] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:00 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:00.337795 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095fb459-c560-47cd-ab3b-a3d288b7dc39 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:00.347359 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 776/1563] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:03:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:03:00.347359 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783c1ee-3b83-4cc5-aecc-29cd3cd152c9 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:00.353657 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095fb459-c560-47cd-ab3b-a3d288b7dc39 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:00.356506 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783c1ee-3b83-4cc5-aecc-29cd3cd152c9 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:00.358280 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d660e-c9ec-4a2a-8768-4725edc641b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:00.358934 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d660e-c9ec-4a2a-8768-4725edc641b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:00.359215 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d660e-c9ec-4a2a-8768-4725edc641b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:00.359518 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d660e-c9ec-4a2a-8768-4725edc641b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:00.360241 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3d660e-c9ec-4a2a-8768-4725edc641b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:03:00.360549 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3d660e-c9ec-4a2a-8768-4725edc641b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:00.361102 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb2b75-1e10-4795-a837-39d911c2f27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:00.361563 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: WARNING py.warnings [None req-4b3d660e-c9ec-4a2a-8768-4725edc641b9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 07 22:03:00.361563 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: warnings.warn(msg) Jun 07 22:03:00.361563 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: Jun 07 22:03:00.361799 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb2b75-1e10-4795-a837-39d911c2f27a admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:00.362012 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb2b75-1e10-4795-a837-39d911c2f27a admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:00.362285 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb2b75-1e10-4795-a837-39d911c2f27a admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:00.362443 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3d660e-c9ec-4a2a-8768-4725edc641b9 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:00.362986 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceb2b75-1e10-4795-a837-39d911c2f27a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:03:00.363288 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceb2b75-1e10-4795-a837-39d911c2f27a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:00.364959 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceb2b75-1e10-4795-a837-39d911c2f27a admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:00.376141 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 788/1564] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:03:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:03:00.377049 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d660e-c9ec-4a2a-8768-4725edc641b9 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:00.383362 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 777/1565] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:03:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:03:00.383362 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb2b75-1e10-4795-a837-39d911c2f27a admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:00.394926 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:00.400405 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d660e-c9ec-4a2a-8768-4725edc641b9 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:00.400739 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb2b75-1e10-4795-a837-39d911c2f27a admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:00.406126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bb2da2-62d3-4f01-8d18-5ea3a878c1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:00.406657 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c521d9-3584-42e9-80f2-97e858c1ccc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:00.406787 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb2da2-62d3-4f01-8d18-5ea3a878c1b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:00.407073 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb2da2-62d3-4f01-8d18-5ea3a878c1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:00.407381 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb2da2-62d3-4f01-8d18-5ea3a878c1b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:00.407495 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c521d9-3584-42e9-80f2-97e858c1ccc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:00.407747 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c521d9-3584-42e9-80f2-97e858c1ccc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:00.408020 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c521d9-3584-42e9-80f2-97e858c1ccc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:00.408125 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bb2da2-62d3-4f01-8d18-5ea3a878c1b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:03:00.408419 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bb2da2-62d3-4f01-8d18-5ea3a878c1b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:00.408749 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c521d9-3584-42e9-80f2-97e858c1ccc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:03:00.409058 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c521d9-3584-42e9-80f2-97e858c1ccc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:00.410010 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bb2da2-62d3-4f01-8d18-5ea3a878c1b8 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:00.410704 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c521d9-3584-42e9-80f2-97e858c1ccc9 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:00.421937 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 778/1566] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:03:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:03:00.422756 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bb2da2-62d3-4f01-8d18-5ea3a878c1b8 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:00.424281 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 789/1567] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:03:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:03:00.425097 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c521d9-3584-42e9-80f2-97e858c1ccc9 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:00.432381 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bb2da2-62d3-4f01-8d18-5ea3a878c1b8 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:00.436466 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c521d9-3584-42e9-80f2-97e858c1ccc9 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:00.437755 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e6f4bb-9677-40c5-90b4-068725b61b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:00.438412 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e6f4bb-9677-40c5-90b4-068725b61b7b admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:00.438701 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e6f4bb-9677-40c5-90b4-068725b61b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:00.438980 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e6f4bb-9677-40c5-90b4-068725b61b7b admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:00.439738 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e6f4bb-9677-40c5-90b4-068725b61b7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:03:00.440047 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e6f4bb-9677-40c5-90b4-068725b61b7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:00.441121 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a8633-5446-4295-84aa-779489b32672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:00.441640 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e6f4bb-9677-40c5-90b4-068725b61b7b admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:00.441776 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a8633-5446-4295-84aa-779489b32672 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:00.442055 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a8633-5446-4295-84aa-779489b32672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:00.442328 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a8633-5446-4295-84aa-779489b32672 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:00.443014 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7a8633-5446-4295-84aa-779489b32672 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:00.444063 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: WARNING py.warnings [None req-ac7a8633-5446-4295-84aa-779489b32672 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 22:03:00.444063 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: warnings.warn(msg) Jun 07 22:03:00.444063 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: Jun 07 22:03:00.445243 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7a8633-5446-4295-84aa-779489b32672 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:00.454836 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 779/1568] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:03:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:03:00.455852 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e6f4bb-9677-40c5-90b4-068725b61b7b admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:00.468580 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e6f4bb-9677-40c5-90b4-068725b61b7b admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:00.473138 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcaefeb-6588-4f77-b929-daa44fdad513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:00.476861 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 790/1569] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:03:00 2022] POST /identity/v3/users => generated 293 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:00.477014 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcaefeb-6588-4f77-b929-daa44fdad513 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:00.477014 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcaefeb-6588-4f77-b929-daa44fdad513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:00.477207 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a8633-5446-4295-84aa-779489b32672 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:00.477313 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcaefeb-6588-4f77-b929-daa44fdad513 admin admin] PATH_INFO: `/v3/projects/771bb741cb47444b8d7c56f9bcb62e93/users/e174947bd6e640ae8169ccb80794abff/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:00.486392 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a8633-5446-4295-84aa-779489b32672 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:00.491001 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e967b97-ed9d-45c0-b321-48aa9ba19b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:00.491714 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e967b97-ed9d-45c0-b321-48aa9ba19b2e admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:00.491998 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e967b97-ed9d-45c0-b321-48aa9ba19b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:00.492117 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcaefeb-6588-4f77-b929-daa44fdad513 admin admin] RBAC: Authorizing `identity:create_grant(project_id=771bb741cb47444b8d7c56f9bcb62e93, user_id=e174947bd6e640ae8169ccb80794abff, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:00.492275 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e967b97-ed9d-45c0-b321-48aa9ba19b2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:00.492963 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e967b97-ed9d-45c0-b321-48aa9ba19b2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:00.493066 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: WARNING py.warnings [None req-3fcaefeb-6588-4f77-b929-daa44fdad513 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 22:03:00.493066 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: warnings.warn(msg) Jun 07 22:03:00.493066 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: Jun 07 22:03:00.494843 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e967b97-ed9d-45c0-b321-48aa9ba19b2e admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:00.494973 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcaefeb-6588-4f77-b929-daa44fdad513 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:00.509524 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 780/1570] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:03:00 2022] PUT /identity/v3/projects/771bb741cb47444b8d7c56f9bcb62e93/users/e174947bd6e640ae8169ccb80794abff/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:00.510311 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcaefeb-6588-4f77-b929-daa44fdad513 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:00.532246 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 791/1571] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:03:00 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:00.533525 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e967b97-ed9d-45c0-b321-48aa9ba19b2e admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:00.576737 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcaefeb-6588-4f77-b929-daa44fdad513 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:00.595819 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e967b97-ed9d-45c0-b321-48aa9ba19b2e admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:00.600302 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a91d1fc-1e89-4435-929f-b0998f7f02c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:00.601221 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a91d1fc-1e89-4435-929f-b0998f7f02c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:00.601506 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a91d1fc-1e89-4435-929f-b0998f7f02c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:00.601785 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a91d1fc-1e89-4435-929f-b0998f7f02c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:00.602470 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a91d1fc-1e89-4435-929f-b0998f7f02c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:00.604408 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a91d1fc-1e89-4435-929f-b0998f7f02c3 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:00.632876 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 792/1572] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:03:00 2022] POST /identity/v3/projects => generated 377 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:00.633864 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a91d1fc-1e89-4435-929f-b0998f7f02c3 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:00.634482 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a566ad44-65c8-4bb2-9f83-9155ea26b248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:00.635154 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a566ad44-65c8-4bb2-9f83-9155ea26b248 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:00.635473 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a566ad44-65c8-4bb2-9f83-9155ea26b248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:00.635753 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a566ad44-65c8-4bb2-9f83-9155ea26b248 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:00.636447 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a566ad44-65c8-4bb2-9f83-9155ea26b248 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:00.639101 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a566ad44-65c8-4bb2-9f83-9155ea26b248 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:00.665578 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 781/1573] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:03:00 2022] POST /identity/v3/projects => generated 371 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:00.666532 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a566ad44-65c8-4bb2-9f83-9155ea26b248 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:00.681431 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:00.697899 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a91d1fc-1e89-4435-929f-b0998f7f02c3 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:00.734148 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a566ad44-65c8-4bb2-9f83-9155ea26b248 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:00.738565 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:00.738863 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f616cb4-9733-410c-9acd-80e73088dad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:00.739557 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f616cb4-9733-410c-9acd-80e73088dad4 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:00.739839 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f616cb4-9733-410c-9acd-80e73088dad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:00.740119 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f616cb4-9733-410c-9acd-80e73088dad4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:00.740810 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f616cb4-9733-410c-9acd-80e73088dad4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:00.742702 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f616cb4-9733-410c-9acd-80e73088dad4 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:00.756827 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be7fda9-7a85-42ed-b1a6-0324186eae37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:00.757637 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be7fda9-7a85-42ed-b1a6-0324186eae37 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:00.758014 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be7fda9-7a85-42ed-b1a6-0324186eae37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:00.758368 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be7fda9-7a85-42ed-b1a6-0324186eae37 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:00.759420 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be7fda9-7a85-42ed-b1a6-0324186eae37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:00.761387 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be7fda9-7a85-42ed-b1a6-0324186eae37 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:00.770725 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 782/1574] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:03:00 2022] POST /identity/v3/projects => generated 373 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:00.774792 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f616cb4-9733-410c-9acd-80e73088dad4 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:00.791124 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 793/1575] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:03:00 2022] POST /identity/v3/projects => generated 391 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:00.792001 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be7fda9-7a85-42ed-b1a6-0324186eae37 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:00.845871 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f616cb4-9733-410c-9acd-80e73088dad4 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:00.850363 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cda54b-0796-4762-8b86-df4d0f5cd159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:00.851312 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cda54b-0796-4762-8b86-df4d0f5cd159 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:00.851618 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cda54b-0796-4762-8b86-df4d0f5cd159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:00.851890 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cda54b-0796-4762-8b86-df4d0f5cd159 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:00.852574 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cda54b-0796-4762-8b86-df4d0f5cd159 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:00.854453 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cda54b-0796-4762-8b86-df4d0f5cd159 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:00.854567 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be7fda9-7a85-42ed-b1a6-0324186eae37 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:00.859841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8477d5-986a-41d2-a134-07ea1d5536ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:00.860508 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8477d5-986a-41d2-a134-07ea1d5536ce admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:00.860795 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8477d5-986a-41d2-a134-07ea1d5536ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:00.861073 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8477d5-986a-41d2-a134-07ea1d5536ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:00.861799 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8477d5-986a-41d2-a134-07ea1d5536ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:00.863409 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8477d5-986a-41d2-a134-07ea1d5536ce admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:00.875082 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 794/1576] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:00 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:00.876359 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa44acb-6045-4d71-bf17-fe7d5629ced9 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:00.876648 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa44acb-6045-4d71-bf17-fe7d5629ced9 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:00.876923 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa44acb-6045-4d71-bf17-fe7d5629ced9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:00.884617 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 783/1577] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:03:00 2022] POST /identity/v3/projects => generated 377 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:00.885608 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cda54b-0796-4762-8b86-df4d0f5cd159 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:00.903167 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-6aa44acb-6045-4d71-bf17-fe7d5629ced9 None None] MFA Rules not processed for user `e174947bd6e640ae8169ccb80794abff`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:00.904506 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-6aa44acb-6045-4d71-bf17-fe7d5629ced9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:00.950278 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cda54b-0796-4762-8b86-df4d0f5cd159 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:00.954878 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce2edd2-f150-4acd-8cf8-e692dbd1c5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:00.955576 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce2edd2-f150-4acd-8cf8-e692dbd1c5b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:00.955860 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce2edd2-f150-4acd-8cf8-e692dbd1c5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:00.956134 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce2edd2-f150-4acd-8cf8-e692dbd1c5b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:00.956814 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce2edd2-f150-4acd-8cf8-e692dbd1c5b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:00.958741 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce2edd2-f150-4acd-8cf8-e692dbd1c5b3 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:00.969117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 795/1578] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:00 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:00.970148 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa44acb-6045-4d71-bf17-fe7d5629ced9 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:00.980247 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 784/1579] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:03:00 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:00.981508 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce2edd2-f150-4acd-8cf8-e692dbd1c5b3 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:00.981644 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa44acb-6045-4d71-bf17-fe7d5629ced9 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:00.986137 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94edcb0e-a311-4f97-ad25-7fde0d34ac5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:00.986804 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94edcb0e-a311-4f97-ad25-7fde0d34ac5e admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:00.987089 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94edcb0e-a311-4f97-ad25-7fde0d34ac5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:00.987396 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94edcb0e-a311-4f97-ad25-7fde0d34ac5e admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:00.988109 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94edcb0e-a311-4f97-ad25-7fde0d34ac5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:00.990017 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94edcb0e-a311-4f97-ad25-7fde0d34ac5e admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:00.991094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce2edd2-f150-4acd-8cf8-e692dbd1c5b3 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:00.995678 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6126ff-28a4-4966-b0fd-f72e38f3be53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:00.996858 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6126ff-28a4-4966-b0fd-f72e38f3be53 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:00.997140 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6126ff-28a4-4966-b0fd-f72e38f3be53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:00.997612 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6126ff-28a4-4966-b0fd-f72e38f3be53 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:00.998294 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6126ff-28a4-4966-b0fd-f72e38f3be53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:01.000243 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6126ff-28a4-4966-b0fd-f72e38f3be53 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:01.010188 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:01.011110 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 796/1580] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:03:00 2022] POST /identity/v3/users => generated 289 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:01.012914 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94edcb0e-a311-4f97-ad25-7fde0d34ac5e admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:01.023428 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94edcb0e-a311-4f97-ad25-7fde0d34ac5e admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:01.027921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 785/1581] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:03:00 2022] POST /identity/v3/users => generated 286 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:01.027921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6126ff-28a4-4966-b0fd-f72e38f3be53 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:01.028401 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e295d8-824a-4a62-a996-56d1e32be372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:01.029117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e295d8-824a-4a62-a996-56d1e32be372 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:01.029406 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e295d8-824a-4a62-a996-56d1e32be372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:01.029681 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e295d8-824a-4a62-a996-56d1e32be372 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:01.030374 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e295d8-824a-4a62-a996-56d1e32be372 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:01.032335 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e295d8-824a-4a62-a996-56d1e32be372 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:01.035318 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6126ff-28a4-4966-b0fd-f72e38f3be53 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:01.040870 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba87f023-0387-4d91-b6bd-d1941aa998a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:01.041699 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba87f023-0387-4d91-b6bd-d1941aa998a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:01.041986 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba87f023-0387-4d91-b6bd-d1941aa998a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:01.042268 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba87f023-0387-4d91-b6bd-d1941aa998a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:01.043926 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba87f023-0387-4d91-b6bd-d1941aa998a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:01.044937 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba87f023-0387-4d91-b6bd-d1941aa998a0 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:01.052194 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 797/1582] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:03:01 2022] POST /identity/v3/users => generated 287 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:01.053013 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e295d8-824a-4a62-a996-56d1e32be372 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:01.064831 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e295d8-824a-4a62-a996-56d1e32be372 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:01.068959 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 786/1583] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:03:01 2022] POST /identity/v3/users => generated 296 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:01.068959 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba87f023-0387-4d91-b6bd-d1941aa998a0 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:01.072571 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d8f9f6-f3cd-4caa-960e-4ca420eeb75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:01.073232 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8f9f6-f3cd-4caa-960e-4ca420eeb75f admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:01.073511 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8f9f6-f3cd-4caa-960e-4ca420eeb75f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:01.073788 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8f9f6-f3cd-4caa-960e-4ca420eeb75f admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:01.075080 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d8f9f6-f3cd-4caa-960e-4ca420eeb75f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:01.076701 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d8f9f6-f3cd-4caa-960e-4ca420eeb75f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:01.077010 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba87f023-0387-4d91-b6bd-d1941aa998a0 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:01.082318 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafc834b-265c-4b51-9d0f-3ae4a32147c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:01.083035 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafc834b-265c-4b51-9d0f-3ae4a32147c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:01.083317 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafc834b-265c-4b51-9d0f-3ae4a32147c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:01.083617 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafc834b-265c-4b51-9d0f-3ae4a32147c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:01.084285 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafc834b-265c-4b51-9d0f-3ae4a32147c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:01.086175 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafc834b-265c-4b51-9d0f-3ae4a32147c6 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:01.087499 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 798/1584] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:01.088141 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d8f9f6-f3cd-4caa-960e-4ca420eeb75f admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:01.098803 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d8f9f6-f3cd-4caa-960e-4ca420eeb75f admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:01.104219 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f073e-cc27-449d-a766-95a11bf24ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:01.104863 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f073e-cc27-449d-a766-95a11bf24ef1 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:01.105152 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f073e-cc27-449d-a766-95a11bf24ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:01.105488 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f073e-cc27-449d-a766-95a11bf24ef1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:01.106339 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4f073e-cc27-449d-a766-95a11bf24ef1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:01.106548 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 787/1585] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:03:01 2022] POST /identity/v3/users => generated 289 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:01.107391 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafc834b-265c-4b51-9d0f-3ae4a32147c6 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:01.108357 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4f073e-cc27-449d-a766-95a11bf24ef1 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:01.112541 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:01.116982 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafc834b-265c-4b51-9d0f-3ae4a32147c6 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:01.122694 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cfec95-22d8-498c-b506-462f7e343545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:01.123392 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cfec95-22d8-498c-b506-462f7e343545 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:01.123685 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cfec95-22d8-498c-b506-462f7e343545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:01.123964 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cfec95-22d8-498c-b506-462f7e343545 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:01.124695 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cfec95-22d8-498c-b506-462f7e343545 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:01.126294 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cfec95-22d8-498c-b506-462f7e343545 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:01.143667 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 799/1586] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:03:01 2022] POST /identity/v3/projects => generated 363 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:01.143848 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 788/1587] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:01.143978 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f073e-cc27-449d-a766-95a11bf24ef1 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:01.144073 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cfec95-22d8-498c-b506-462f7e343545 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:01.153205 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cfec95-22d8-498c-b506-462f7e343545 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:01.203718 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f073e-cc27-449d-a766-95a11bf24ef1 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:01.208940 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be252c3e-eeba-4012-84d5-38af56551841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:01.209605 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be252c3e-eeba-4012-84d5-38af56551841 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:01.209885 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be252c3e-eeba-4012-84d5-38af56551841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:01.210153 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be252c3e-eeba-4012-84d5-38af56551841 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:01.210899 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be252c3e-eeba-4012-84d5-38af56551841 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:01.211997 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0810563b-fe41-4b3a-a4d9-918bf826a576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:01.212521 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be252c3e-eeba-4012-84d5-38af56551841 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:01.212696 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0810563b-fe41-4b3a-a4d9-918bf826a576 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:01.212954 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0810563b-fe41-4b3a-a4d9-918bf826a576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:01.213221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0810563b-fe41-4b3a-a4d9-918bf826a576 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:01.213972 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0810563b-fe41-4b3a-a4d9-918bf826a576 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:01.215761 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0810563b-fe41-4b3a-a4d9-918bf826a576 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:01.222585 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 800/1588] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:01.223413 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be252c3e-eeba-4012-84d5-38af56551841 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:01.228868 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 789/1589] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:01.229835 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0810563b-fe41-4b3a-a4d9-918bf826a576 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:01.236552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be252c3e-eeba-4012-84d5-38af56551841 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:01.239420 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0810563b-fe41-4b3a-a4d9-918bf826a576 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:01.241164 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a48b8e-03c8-47de-b09b-97f59869c8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:01.241821 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a48b8e-03c8-47de-b09b-97f59869c8bf admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:01.242105 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a48b8e-03c8-47de-b09b-97f59869c8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:01.242382 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a48b8e-03c8-47de-b09b-97f59869c8bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:01.243135 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a48b8e-03c8-47de-b09b-97f59869c8bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:01.243995 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2412289-25a6-4b93-b2fa-c9f7ea8c9e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:01.244683 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2412289-25a6-4b93-b2fa-c9f7ea8c9e02 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:01.244798 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a48b8e-03c8-47de-b09b-97f59869c8bf admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:01.244968 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2412289-25a6-4b93-b2fa-c9f7ea8c9e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:01.245242 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2412289-25a6-4b93-b2fa-c9f7ea8c9e02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:01.246443 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2412289-25a6-4b93-b2fa-c9f7ea8c9e02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:01.248067 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2412289-25a6-4b93-b2fa-c9f7ea8c9e02 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:01.255289 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 801/1590] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:01.256084 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a48b8e-03c8-47de-b09b-97f59869c8bf admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:01.260378 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 790/1591] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:01.260378 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2412289-25a6-4b93-b2fa-c9f7ea8c9e02 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:01.266189 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a48b8e-03c8-47de-b09b-97f59869c8bf admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:01.270248 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2412289-25a6-4b93-b2fa-c9f7ea8c9e02 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:01.270922 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a0468-3353-42d2-9baa-d5d49a390e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:01.271826 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a0468-3353-42d2-9baa-d5d49a390e83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:01.272106 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a0468-3353-42d2-9baa-d5d49a390e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:01.272375 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a0468-3353-42d2-9baa-d5d49a390e83 admin admin] PATH_INFO: `/v3/projects/49d73c22e28a450a99d0952aa686dccf/users/607b70b4d6e14c95ae2880dd3577c8ae/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:01.274801 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a4578-8102-424a-a2d3-d4c488f602d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:01.275457 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a4578-8102-424a-a2d3-d4c488f602d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:01.275744 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a4578-8102-424a-a2d3-d4c488f602d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:01.276018 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a4578-8102-424a-a2d3-d4c488f602d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:01.276720 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318a4578-8102-424a-a2d3-d4c488f602d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:01.278372 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318a4578-8102-424a-a2d3-d4c488f602d3 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:01.288067 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8a0468-3353-42d2-9baa-d5d49a390e83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49d73c22e28a450a99d0952aa686dccf, user_id=607b70b4d6e14c95ae2880dd3577c8ae, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:01.288391 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 791/1592] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:01.289153 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a4578-8102-424a-a2d3-d4c488f602d3 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:01.291641 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8a0468-3353-42d2-9baa-d5d49a390e83 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:01.299577 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a4578-8102-424a-a2d3-d4c488f602d3 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:01.305528 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 802/1593] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:03:01 2022] PUT /identity/v3/projects/49d73c22e28a450a99d0952aa686dccf/users/607b70b4d6e14c95ae2880dd3577c8ae/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:01.306328 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a0468-3353-42d2-9baa-d5d49a390e83 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:01.356747 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a129908-a250-4c92-bcba-b83f801f8bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:01.357341 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a129908-a250-4c92-bcba-b83f801f8bdd admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:01.357623 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a129908-a250-4c92-bcba-b83f801f8bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:01.357901 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a129908-a250-4c92-bcba-b83f801f8bdd admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:01.358584 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a129908-a250-4c92-bcba-b83f801f8bdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:01.360561 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a129908-a250-4c92-bcba-b83f801f8bdd admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:01.369994 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a0468-3353-42d2-9baa-d5d49a390e83 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:01.376605 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398f66f9-bfd2-4a12-9c3a-166d516f25c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:01.377262 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f66f9-bfd2-4a12-9c3a-166d516f25c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:01.377543 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f66f9-bfd2-4a12-9c3a-166d516f25c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:01.377809 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f66f9-bfd2-4a12-9c3a-166d516f25c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:01.378560 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398f66f9-bfd2-4a12-9c3a-166d516f25c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:01.380297 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398f66f9-bfd2-4a12-9c3a-166d516f25c3 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:01.382542 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 792/1594] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:03:01 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:01.383450 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a129908-a250-4c92-bcba-b83f801f8bdd admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:01.391166 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 803/1595] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:01.393523 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a129908-a250-4c92-bcba-b83f801f8bdd admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:01.394866 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398f66f9-bfd2-4a12-9c3a-166d516f25c3 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:01.398334 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef211dc-f140-46f6-8c22-9c8e0475033a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:01.398989 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef211dc-f140-46f6-8c22-9c8e0475033a admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:01.399273 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef211dc-f140-46f6-8c22-9c8e0475033a admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:01.399638 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef211dc-f140-46f6-8c22-9c8e0475033a admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:01.400377 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef211dc-f140-46f6-8c22-9c8e0475033a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:01.401972 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef211dc-f140-46f6-8c22-9c8e0475033a admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:01.404948 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398f66f9-bfd2-4a12-9c3a-166d516f25c3 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:01.410957 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e1dbf5-8b93-4228-a255-322f83c72ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:01.411773 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e1dbf5-8b93-4228-a255-322f83c72ffa admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:01.412064 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e1dbf5-8b93-4228-a255-322f83c72ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:01.412353 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e1dbf5-8b93-4228-a255-322f83c72ffa admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:01.413083 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e1dbf5-8b93-4228-a255-322f83c72ffa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:01.414688 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e1dbf5-8b93-4228-a255-322f83c72ffa admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:01.415264 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 793/1596] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:01.416128 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef211dc-f140-46f6-8c22-9c8e0475033a admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:01.423286 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:01.428147 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef211dc-f140-46f6-8c22-9c8e0475033a admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:01.435706 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 804/1597] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:01.435706 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e1dbf5-8b93-4228-a255-322f83c72ffa admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:01.435987 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6776839-19a1-47c2-9917-1812f3ccd300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:01.436108 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6776839-19a1-47c2-9917-1812f3ccd300 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:01.436402 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6776839-19a1-47c2-9917-1812f3ccd300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:01.436651 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6776839-19a1-47c2-9917-1812f3ccd300 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:01.437414 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6776839-19a1-47c2-9917-1812f3ccd300 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:01.439003 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6776839-19a1-47c2-9917-1812f3ccd300 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:01.443769 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e1dbf5-8b93-4228-a255-322f83c72ffa admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:01.448412 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b13288-2efa-4c35-84d3-3098a23cafbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:01.449112 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b13288-2efa-4c35-84d3-3098a23cafbb admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:01.449597 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b13288-2efa-4c35-84d3-3098a23cafbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:01.449936 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b13288-2efa-4c35-84d3-3098a23cafbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:01.450100 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 794/1598] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:01.450667 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b13288-2efa-4c35-84d3-3098a23cafbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:01.450886 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6776839-19a1-47c2-9917-1812f3ccd300 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:01.452750 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b13288-2efa-4c35-84d3-3098a23cafbb admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:01.462709 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6776839-19a1-47c2-9917-1812f3ccd300 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:01.467901 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 805/1599] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:01.467901 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64705e86-0f9a-4918-9353-4dbf0e89cd26 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:01.467901 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64705e86-0f9a-4918-9353-4dbf0e89cd26 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:01.467901 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64705e86-0f9a-4918-9353-4dbf0e89cd26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:01.470769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078331fa-49ca-45a2-b733-f5db7008bdf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:01.471468 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078331fa-49ca-45a2-b733-f5db7008bdf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:01.471756 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078331fa-49ca-45a2-b733-f5db7008bdf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:01.472031 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078331fa-49ca-45a2-b733-f5db7008bdf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:01.472756 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078331fa-49ca-45a2-b733-f5db7008bdf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:01.474374 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078331fa-49ca-45a2-b733-f5db7008bdf6 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:01.484902 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-64705e86-0f9a-4918-9353-4dbf0e89cd26 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:01.485745 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 795/1600] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:01.486117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-64705e86-0f9a-4918-9353-4dbf0e89cd26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:01.486529 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078331fa-49ca-45a2-b733-f5db7008bdf6 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:01.494041 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 806/1601] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:01.497093 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64705e86-0f9a-4918-9353-4dbf0e89cd26 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:01.497611 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078331fa-49ca-45a2-b733-f5db7008bdf6 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:01.502595 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea8999-7dad-4586-bd04-6ea4a3448f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:01.503269 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea8999-7dad-4586-bd04-6ea4a3448f79 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:01.504198 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea8999-7dad-4586-bd04-6ea4a3448f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:01.504635 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea8999-7dad-4586-bd04-6ea4a3448f79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:01.505218 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceea8999-7dad-4586-bd04-6ea4a3448f79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:01.506866 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceea8999-7dad-4586-bd04-6ea4a3448f79 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:01.507406 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64705e86-0f9a-4918-9353-4dbf0e89cd26 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:01.512777 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11a5cd3-54b5-4e61-b12a-326c18f70018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:01.513768 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a5cd3-54b5-4e61-b12a-326c18f70018 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:01.514060 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a5cd3-54b5-4e61-b12a-326c18f70018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:01.514341 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a5cd3-54b5-4e61-b12a-326c18f70018 admin admin] PATH_INFO: `/v3/projects/3ec97d2dae1f45ed82d63ba7a748cd7d/users/9fe11b29cc0e4a78bddc435763a79adb/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:01.517468 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 796/1602] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:01.518241 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea8999-7dad-4586-bd04-6ea4a3448f79 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:01.526458 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:01.531025 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea8999-7dad-4586-bd04-6ea4a3448f79 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:01.531772 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11a5cd3-54b5-4e61-b12a-326c18f70018 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ec97d2dae1f45ed82d63ba7a748cd7d, user_id=9fe11b29cc0e4a78bddc435763a79adb, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:01.534888 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11a5cd3-54b5-4e61-b12a-326c18f70018 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:01.535667 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c8d7f0-baa4-443f-8b2c-8ccc76d15a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:01.536546 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8d7f0-baa4-443f-8b2c-8ccc76d15a51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:01.536828 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8d7f0-baa4-443f-8b2c-8ccc76d15a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:01.537112 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8d7f0-baa4-443f-8b2c-8ccc76d15a51 admin admin] PATH_INFO: `/v3/projects/7e8d5b4fdf9a48a296359e167017123f/users/1378a898f9534635857e5958d32f00c5/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:01.548441 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 807/1603] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:03:01 2022] PUT /identity/v3/projects/3ec97d2dae1f45ed82d63ba7a748cd7d/users/9fe11b29cc0e4a78bddc435763a79adb/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:01.556935 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11a5cd3-54b5-4e61-b12a-326c18f70018 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:01.563170 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c8d7f0-baa4-443f-8b2c-8ccc76d15a51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e8d5b4fdf9a48a296359e167017123f, user_id=1378a898f9534635857e5958d32f00c5, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:01.565925 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c8d7f0-baa4-443f-8b2c-8ccc76d15a51 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:01.579204 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 797/1604] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:03:01 2022] PUT /identity/v3/projects/7e8d5b4fdf9a48a296359e167017123f/users/1378a898f9534635857e5958d32f00c5/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:01.583572 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c8d7f0-baa4-443f-8b2c-8ccc76d15a51 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:01.623504 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11a5cd3-54b5-4e61-b12a-326c18f70018 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:01.647391 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c8d7f0-baa4-443f-8b2c-8ccc76d15a51 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:01.652028 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976bee0a-8470-4c35-9cdc-6c1b245123d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:01.653501 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976bee0a-8470-4c35-9cdc-6c1b245123d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:01.653783 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976bee0a-8470-4c35-9cdc-6c1b245123d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:01.654062 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976bee0a-8470-4c35-9cdc-6c1b245123d0 admin admin] PATH_INFO: `/v3/projects/4637b625bf8245038c9606f3e4b0bcd4/users/17ce5e31e7724b77ab47c52ee0d3021e/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:01.668365 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976bee0a-8470-4c35-9cdc-6c1b245123d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4637b625bf8245038c9606f3e4b0bcd4, user_id=17ce5e31e7724b77ab47c52ee0d3021e, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:01.671022 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976bee0a-8470-4c35-9cdc-6c1b245123d0 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:01.680134 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff3b42c-7bf5-4676-9149-d687d7e04894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:01.681107 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff3b42c-7bf5-4676-9149-d687d7e04894 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:01.681419 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff3b42c-7bf5-4676-9149-d687d7e04894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:01.681702 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff3b42c-7bf5-4676-9149-d687d7e04894 admin admin] PATH_INFO: `/v3/projects/0a9c7772d09c4827a38b523447736134/users/9164a923819d4d34a679064e7f4b44d1/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:01.685766 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 798/1605] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:03:01 2022] PUT /identity/v3/projects/4637b625bf8245038c9606f3e4b0bcd4/users/17ce5e31e7724b77ab47c52ee0d3021e/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:01.686829 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976bee0a-8470-4c35-9cdc-6c1b245123d0 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:01.704831 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff3b42c-7bf5-4676-9149-d687d7e04894 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a9c7772d09c4827a38b523447736134, user_id=9164a923819d4d34a679064e7f4b44d1, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:01.707654 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff3b42c-7bf5-4676-9149-d687d7e04894 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:01.721120 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 808/1606] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:03:01 2022] PUT /identity/v3/projects/0a9c7772d09c4827a38b523447736134/users/9164a923819d4d34a679064e7f4b44d1/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:01.723568 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff3b42c-7bf5-4676-9149-d687d7e04894 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:01.762627 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976bee0a-8470-4c35-9cdc-6c1b245123d0 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:01.786050 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff3b42c-7bf5-4676-9149-d687d7e04894 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:01.791581 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e871b7-e565-4f08-afc0-310de6b6c083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:01.792495 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e871b7-e565-4f08-afc0-310de6b6c083 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:01.792799 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e871b7-e565-4f08-afc0-310de6b6c083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:01.793072 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e871b7-e565-4f08-afc0-310de6b6c083 admin admin] PATH_INFO: `/v3/projects/d32f58b7303541388483ca4b881a552e/users/0e68eff6a08d4193b3167d47f4e1f97c/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:01.796371 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:01.809106 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e871b7-e565-4f08-afc0-310de6b6c083 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d32f58b7303541388483ca4b881a552e, user_id=0e68eff6a08d4193b3167d47f4e1f97c, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:01.813900 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e871b7-e565-4f08-afc0-310de6b6c083 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:01.825247 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ab173-5c8d-48a1-bd83-9c3e7930e5b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:01.826152 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ab173-5c8d-48a1-bd83-9c3e7930e5b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:01.826462 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ab173-5c8d-48a1-bd83-9c3e7930e5b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:01.826742 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ab173-5c8d-48a1-bd83-9c3e7930e5b9 admin admin] PATH_INFO: `/v3/projects/0a4fc48a5efe4d8b974757a5bf7d947e/users/c44c1788959e4b339c9cd5aad4318496/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:01.828526 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 809/1607] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:03:01 2022] PUT /identity/v3/projects/d32f58b7303541388483ca4b881a552e/users/0e68eff6a08d4193b3167d47f4e1f97c/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:01.829436 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e871b7-e565-4f08-afc0-310de6b6c083 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:01.835940 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:01.842712 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8ab173-5c8d-48a1-bd83-9c3e7930e5b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a4fc48a5efe4d8b974757a5bf7d947e, user_id=c44c1788959e4b339c9cd5aad4318496, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:01.845390 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8ab173-5c8d-48a1-bd83-9c3e7930e5b9 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:01.861291 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 799/1608] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:03:01 2022] PUT /identity/v3/projects/0a4fc48a5efe4d8b974757a5bf7d947e/users/c44c1788959e4b339c9cd5aad4318496/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:01.862083 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ab173-5c8d-48a1-bd83-9c3e7930e5b9 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:01.893958 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e871b7-e565-4f08-afc0-310de6b6c083 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:01.924588 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ab173-5c8d-48a1-bd83-9c3e7930e5b9 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:01.929260 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff53d42-65ab-46a8-b42d-62325ec61680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:01.930147 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff53d42-65ab-46a8-b42d-62325ec61680 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:01.930433 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff53d42-65ab-46a8-b42d-62325ec61680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:01.930712 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff53d42-65ab-46a8-b42d-62325ec61680 admin admin] PATH_INFO: `/v3/projects/0cd1fa7e85f54b228b5015a172ffb10e/users/b8ad47be8834455c9e0a70f453f8fbec/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:01.944441 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d24b1-e182-45bc-a592-fb57a4d1c856 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:01.945087 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d24b1-e182-45bc-a592-fb57a4d1c856 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:01.945367 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d24b1-e182-45bc-a592-fb57a4d1c856 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:01.945638 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d24b1-e182-45bc-a592-fb57a4d1c856 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:01.946519 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff53d42-65ab-46a8-b42d-62325ec61680 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cd1fa7e85f54b228b5015a172ffb10e, user_id=b8ad47be8834455c9e0a70f453f8fbec, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:01.949291 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff53d42-65ab-46a8-b42d-62325ec61680 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:01.956480 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5d24b1-e182-45bc-a592-fb57a4d1c856 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:01.958078 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5d24b1-e182-45bc-a592-fb57a4d1c856 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:01.963446 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 800/1609] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:03:01 2022] PUT /identity/v3/projects/0cd1fa7e85f54b228b5015a172ffb10e/users/b8ad47be8834455c9e0a70f453f8fbec/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:01.965611 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59076783-a25f-48f3-a46a-88192f6ad36c None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:01.965898 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59076783-a25f-48f3-a46a-88192f6ad36c None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:01.966181 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59076783-a25f-48f3-a46a-88192f6ad36c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:01.969127 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 810/1610] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:03:01.970476 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea22b99-52a3-4d97-9262-70f8fc8b91e5 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:01.970770 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea22b99-52a3-4d97-9262-70f8fc8b91e5 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:01.971218 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea22b99-52a3-4d97-9262-70f8fc8b91e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:01.990582 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-7ea22b99-52a3-4d97-9262-70f8fc8b91e5 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:01.992157 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-7ea22b99-52a3-4d97-9262-70f8fc8b91e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:01.999086 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-59076783-a25f-48f3-a46a-88192f6ad36c None None] MFA Rules not processed for user `9fe11b29cc0e4a78bddc435763a79adb`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:02.000528 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-59076783-a25f-48f3-a46a-88192f6ad36c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:02.065907 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 811/1611] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:02.067251 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50509097-373c-42c3-8634-a371b89c0851 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:02.067572 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50509097-373c-42c3-8634-a371b89c0851 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:02.067852 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50509097-373c-42c3-8634-a371b89c0851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:02.087578 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 801/1612] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:02.088968 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d79b7f-1339-4632-a19b-f050f78c420c None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:02.089258 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d79b7f-1339-4632-a19b-f050f78c420c None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:02.089541 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d79b7f-1339-4632-a19b-f050f78c420c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:02.104975 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-50509097-373c-42c3-8634-a371b89c0851 None None] MFA Rules not processed for user `17ce5e31e7724b77ab47c52ee0d3021e`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:02.106432 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-50509097-373c-42c3-8634-a371b89c0851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:02.140364 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-42d79b7f-1339-4632-a19b-f050f78c420c None None] MFA Rules not processed for user `9164a923819d4d34a679064e7f4b44d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:02.142433 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-42d79b7f-1339-4632-a19b-f050f78c420c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:02.165906 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:02.171111 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:02.195227 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 812/1613] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:02.196577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835796b-8627-4157-a3a8-47b2b3dd39fc None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:02.196781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835796b-8627-4157-a3a8-47b2b3dd39fc None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:02.197058 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835796b-8627-4157-a3a8-47b2b3dd39fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:02.219646 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-3835796b-8627-4157-a3a8-47b2b3dd39fc None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:02.219646 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-3835796b-8627-4157-a3a8-47b2b3dd39fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:02.227943 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 813/1614] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:02.233081 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 802/1615] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:02.233081 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d79b7f-1339-4632-a19b-f050f78c420c None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:02.239014 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36c729-fe72-4d6e-81e8-39e49ba4f061 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:02.239305 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36c729-fe72-4d6e-81e8-39e49ba4f061 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:02.239612 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36c729-fe72-4d6e-81e8-39e49ba4f061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:02.245585 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d79b7f-1339-4632-a19b-f050f78c420c None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:02.253234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d154a0-f39c-4b8d-8a10-fb6c07021e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:02.254274 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d154a0-f39c-4b8d-8a10-fb6c07021e34 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:02.254563 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d154a0-f39c-4b8d-8a10-fb6c07021e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:02.254838 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d154a0-f39c-4b8d-8a10-fb6c07021e34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:02.255636 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d154a0-f39c-4b8d-8a10-fb6c07021e34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:02.257264 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d154a0-f39c-4b8d-8a10-fb6c07021e34 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:02.260078 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-1c36c729-fe72-4d6e-81e8-39e49ba4f061 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:02.262400 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-1c36c729-fe72-4d6e-81e8-39e49ba4f061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:02.270491 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 803/1616] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:02.270773 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 814/1617] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:02.271596 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36c729-fe72-4d6e-81e8-39e49ba4f061 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:02.272984 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-1c36c729-fe72-4d6e-81e8-39e49ba4f061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:02.273255 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73860c2-8820-4d7c-94e2-381ffe7dad32 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:02.274016 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73860c2-8820-4d7c-94e2-381ffe7dad32 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:02.274428 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73860c2-8820-4d7c-94e2-381ffe7dad32 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:02.276020 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 804/1618] 10.222.0.13 () {54 vars in 961 bytes} [Tue Jun 7 22:03:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 22:03:02.276714 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73860c2-8820-4d7c-94e2-381ffe7dad32 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:02.293518 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73860c2-8820-4d7c-94e2-381ffe7dad32 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:02.298553 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514235e0-63d3-45e2-a31b-34fbd76d98cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:02.299472 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514235e0-63d3-45e2-a31b-34fbd76d98cc admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:02.299634 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514235e0-63d3-45e2-a31b-34fbd76d98cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:02.299966 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514235e0-63d3-45e2-a31b-34fbd76d98cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:02.300728 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514235e0-63d3-45e2-a31b-34fbd76d98cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:02.302697 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514235e0-63d3-45e2-a31b-34fbd76d98cc admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:02.331824 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 805/1619] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:03:02 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:02.335684 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514235e0-63d3-45e2-a31b-34fbd76d98cc admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:02.341313 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36c729-fe72-4d6e-81e8-39e49ba4f061 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:02.396841 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514235e0-63d3-45e2-a31b-34fbd76d98cc admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:02.402461 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc9279-6958-4ca2-a234-2f9c9fce1a51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:02.403308 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc9279-6958-4ca2-a234-2f9c9fce1a51 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:02.403801 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a43af-a437-4e8e-86c4-509a82fd308e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:02.403950 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc9279-6958-4ca2-a234-2f9c9fce1a51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:02.404237 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc9279-6958-4ca2-a234-2f9c9fce1a51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:02.404676 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a43af-a437-4e8e-86c4-509a82fd308e service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:02.405031 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a43af-a437-4e8e-86c4-509a82fd308e service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:02.405409 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a43af-a437-4e8e-86c4-509a82fd308e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:02.473108 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a43af-a437-4e8e-86c4-509a82fd308e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:02.474563 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a43af-a437-4e8e-86c4-509a82fd308e service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:02.477891 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abc9279-6958-4ca2-a234-2f9c9fce1a51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:02.479143 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abc9279-6958-4ca2-a234-2f9c9fce1a51 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:02.486765 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 815/1620] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:03:02.488182 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d64b0-45d6-4905-ab40-8c613c164801 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:02.488857 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d64b0-45d6-4905-ab40-8c613c164801 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:02.489181 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d64b0-45d6-4905-ab40-8c613c164801 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:02.490723 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 816/1621] 10.222.0.13 () {54 vars in 961 bytes} [Tue Jun 7 22:03:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 22:03:02.491529 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782d64b0-45d6-4905-ab40-8c613c164801 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:02.495870 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 806/1622] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:03:02.497404 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26231c6e-cd93-4a28-b82b-96c71770126d None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:02.497704 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26231c6e-cd93-4a28-b82b-96c71770126d None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:02.497989 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26231c6e-cd93-4a28-b82b-96c71770126d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:02.507887 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782d64b0-45d6-4905-ab40-8c613c164801 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:02.514852 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af7cacb-e645-46bc-a7d8-4e61084bd845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:02.516010 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7cacb-e645-46bc-a7d8-4e61084bd845 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:02.516456 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7cacb-e645-46bc-a7d8-4e61084bd845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:02.516883 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7cacb-e645-46bc-a7d8-4e61084bd845 admin admin] PATH_INFO: `/v3/projects/0cd1fa7e85f54b228b5015a172ffb10e/users/b8ad47be8834455c9e0a70f453f8fbec/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:02.517757 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-26231c6e-cd93-4a28-b82b-96c71770126d None None] MFA Rules not processed for user `dbe0303ea0764347bfb054870ebba41f`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:02.518979 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-26231c6e-cd93-4a28-b82b-96c71770126d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:02.519499 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af7cacb-e645-46bc-a7d8-4e61084bd845 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cd1fa7e85f54b228b5015a172ffb10e, user_id=b8ad47be8834455c9e0a70f453f8fbec, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:02.522524 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af7cacb-e645-46bc-a7d8-4e61084bd845 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:02.549482 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 817/1623] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:03:02 2022] PUT /identity/v3/projects/0cd1fa7e85f54b228b5015a172ffb10e/users/b8ad47be8834455c9e0a70f453f8fbec/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:02.550534 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af7cacb-e645-46bc-a7d8-4e61084bd845 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:02.570603 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:02.593588 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:02.610977 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 807/1624] 10.222.0.13 () {58 vars in 1058 bytes} [Tue Jun 7 22:03:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:02.611956 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26231c6e-cd93-4a28-b82b-96c71770126d None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:02.622637 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26231c6e-cd93-4a28-b82b-96c71770126d None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:02.623067 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af7cacb-e645-46bc-a7d8-4e61084bd845 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:02.629185 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15a45b-49de-4eae-aabe-cfae963268ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:02.629934 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15a45b-49de-4eae-aabe-cfae963268ff service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:02.630224 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15a45b-49de-4eae-aabe-cfae963268ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:02.630560 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15a45b-49de-4eae-aabe-cfae963268ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:02.692785 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb2a4b5-7699-4f00-85bb-4916dd832f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:02.693492 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2a4b5-7699-4f00-85bb-4916dd832f8c admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:02.693777 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2a4b5-7699-4f00-85bb-4916dd832f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:02.694635 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2a4b5-7699-4f00-85bb-4916dd832f8c admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:02.695309 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb2a4b5-7699-4f00-85bb-4916dd832f8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:02.697259 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb2a4b5-7699-4f00-85bb-4916dd832f8c admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:02.701090 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad15a45b-49de-4eae-aabe-cfae963268ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:02.702635 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad15a45b-49de-4eae-aabe-cfae963268ff service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:02.713808 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 818/1625] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:03:02.715990 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ca1fe-a211-4ef6-ba1b-ec69c9e62914 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:02.716282 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ca1fe-a211-4ef6-ba1b-ec69c9e62914 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:02.716558 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ca1fe-a211-4ef6-ba1b-ec69c9e62914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:02.723877 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 808/1626] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:03:02 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:02.723877 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413aada4-774b-480f-aefb-5d97a162e556 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:02.723877 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413aada4-774b-480f-aefb-5d97a162e556 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:02.723877 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413aada4-774b-480f-aefb-5d97a162e556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:02.740159 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-e86ca1fe-a211-4ef6-ba1b-ec69c9e62914 None None] MFA Rules not processed for user `dbe0303ea0764347bfb054870ebba41f`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:02.741495 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-e86ca1fe-a211-4ef6-ba1b-ec69c9e62914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:02.758574 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-413aada4-774b-480f-aefb-5d97a162e556 None None] MFA Rules not processed for user `b8ad47be8834455c9e0a70f453f8fbec`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:02.760094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-413aada4-774b-480f-aefb-5d97a162e556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:02.826075 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 819/1627] 10.222.0.13 () {58 vars in 1058 bytes} [Tue Jun 7 22:03:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:02.827257 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86ca1fe-a211-4ef6-ba1b-ec69c9e62914 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:02.840879 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86ca1fe-a211-4ef6-ba1b-ec69c9e62914 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:02.847069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d4180-f56b-4c57-a385-5d381c952757 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:02.848031 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d4180-f56b-4c57-a385-5d381c952757 service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:02.848435 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d4180-f56b-4c57-a385-5d381c952757 service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:02.848654 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d4180-f56b-4c57-a385-5d381c952757 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:02.853609 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 809/1628] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:02 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:02.854514 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413aada4-774b-480f-aefb-5d97a162e556 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:02.873588 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413aada4-774b-480f-aefb-5d97a162e556 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:02.879870 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783ab3a-eb5c-4a4b-8f0c-2ec614848007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:02.880964 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783ab3a-eb5c-4a4b-8f0c-2ec614848007 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:02.881256 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783ab3a-eb5c-4a4b-8f0c-2ec614848007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:02.881737 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783ab3a-eb5c-4a4b-8f0c-2ec614848007 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:02.882530 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d783ab3a-eb5c-4a4b-8f0c-2ec614848007 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:02.884257 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d783ab3a-eb5c-4a4b-8f0c-2ec614848007 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:02.895391 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 810/1629] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:02.896183 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783ab3a-eb5c-4a4b-8f0c-2ec614848007 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:02.910767 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783ab3a-eb5c-4a4b-8f0c-2ec614848007 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:02.917604 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ea51a-3673-4b95-8d10-ae645f444241 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:02.917604 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ea51a-3673-4b95-8d10-ae645f444241 service nova] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:02.918460 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ea51a-3673-4b95-8d10-ae645f444241 service nova] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:02.918712 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ea51a-3673-4b95-8d10-ae645f444241 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:02.920149 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899d4180-f56b-4c57-a385-5d381c952757 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:02.922045 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899d4180-f56b-4c57-a385-5d381c952757 service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:02.932750 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 820/1630] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:02 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:03:02.933816 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d4180-f56b-4c57-a385-5d381c952757 service nova] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:02.951912 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:02.955813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:02.997474 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04ea51a-3673-4b95-8d10-ae645f444241 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:02.999414 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04ea51a-3673-4b95-8d10-ae645f444241 service nova] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:03.023149 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 811/1631] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:02 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:03:03.023940 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d4180-f56b-4c57-a385-5d381c952757 service nova] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:03.024612 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ea51a-3673-4b95-8d10-ae645f444241 service nova] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:03.029178 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d9dc1e-55b6-4300-9ec7-d6d6ac026fb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:03.030106 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9dc1e-55b6-4300-9ec7-d6d6ac026fb9 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:03.030614 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9dc1e-55b6-4300-9ec7-d6d6ac026fb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:03.031088 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9dc1e-55b6-4300-9ec7-d6d6ac026fb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:03.044882 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d9dc1e-55b6-4300-9ec7-d6d6ac026fb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:03.047279 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d9dc1e-55b6-4300-9ec7-d6d6ac026fb9 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:03.048885 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ea51a-3673-4b95-8d10-ae645f444241 service nova] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:03.060236 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 821/1632] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:02 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:03:03.062216 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3660b26b-e0f6-4d36-9888-e3676bbb96f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:03.063872 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3660b26b-e0f6-4d36-9888-e3676bbb96f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:03.064199 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3660b26b-e0f6-4d36-9888-e3676bbb96f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:03.064512 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3660b26b-e0f6-4d36-9888-e3676bbb96f8 admin admin] PATH_INFO: `/v3/projects/c9c5640ce3ba4ab5aa7029802fb23cce/users/e78d9898f14c4b7987a9e7d3b4c3b164/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:03.101599 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3660b26b-e0f6-4d36-9888-e3676bbb96f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9c5640ce3ba4ab5aa7029802fb23cce, user_id=e78d9898f14c4b7987a9e7d3b4c3b164, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:03.109498 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3660b26b-e0f6-4d36-9888-e3676bbb96f8 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:03.134062 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 812/1633] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:03:03 2022] PUT /identity/v3/projects/c9c5640ce3ba4ab5aa7029802fb23cce/users/e78d9898f14c4b7987a9e7d3b4c3b164/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:03.138479 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d9dc1e-55b6-4300-9ec7-d6d6ac026fb9 service cinder] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:03.151938 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d9dc1e-55b6-4300-9ec7-d6d6ac026fb9 service cinder] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:03.297840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe024f-0b46-4f34-9335-38dc93858e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:03.298706 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe024f-0b46-4f34-9335-38dc93858e34 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:03.299152 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe024f-0b46-4f34-9335-38dc93858e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:03.300558 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe024f-0b46-4f34-9335-38dc93858e34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:03.301534 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abe024f-0b46-4f34-9335-38dc93858e34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:03.303341 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abe024f-0b46-4f34-9335-38dc93858e34 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:03.316943 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f00be1-1bce-4ed5-903a-fea4a76d7678 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:03.317445 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f00be1-1bce-4ed5-903a-fea4a76d7678 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:03.317905 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f00be1-1bce-4ed5-903a-fea4a76d7678 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:03.319829 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 813/1634] 10.222.0.13 () {54 vars in 912 bytes} [Tue Jun 7 22:03:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 22:03:03.327958 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ac827-ee0f-4190-87f4-233c791911c9 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:03.328287 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ac827-ee0f-4190-87f4-233c791911c9 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:03.328542 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ac827-ee0f-4190-87f4-233c791911c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:03.341077 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 822/1635] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:03 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:03.342440 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe024f-0b46-4f34-9335-38dc93858e34 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:03.370083 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe024f-0b46-4f34-9335-38dc93858e34 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:03.374573 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-d03ac827-ee0f-4190-87f4-233c791911c9 None None] MFA Rules not processed for user `dbe0303ea0764347bfb054870ebba41f`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:03.375815 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-d03ac827-ee0f-4190-87f4-233c791911c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:03.383394 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ea1810-17b5-4d57-808f-6587e024cd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:03.384338 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea1810-17b5-4d57-808f-6587e024cd91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:03.384628 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea1810-17b5-4d57-808f-6587e024cd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:03.385305 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea1810-17b5-4d57-808f-6587e024cd91 admin admin] PATH_INFO: `/v3/projects/c9c5640ce3ba4ab5aa7029802fb23cce/users/e78d9898f14c4b7987a9e7d3b4c3b164/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:03.390813 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ea1810-17b5-4d57-808f-6587e024cd91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9c5640ce3ba4ab5aa7029802fb23cce, user_id=e78d9898f14c4b7987a9e7d3b4c3b164, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:03.399098 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ea1810-17b5-4d57-808f-6587e024cd91 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:03.429943 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 823/1636] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:03:03 2022] PUT /identity/v3/projects/c9c5640ce3ba4ab5aa7029802fb23cce/users/e78d9898f14c4b7987a9e7d3b4c3b164/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:03.431307 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e8955-1b55-4177-89d6-0372225986b3 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:03.431618 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e8955-1b55-4177-89d6-0372225986b3 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:03.431888 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e8955-1b55-4177-89d6-0372225986b3 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:03.433138 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 824/1637] 10.222.0.13 () {54 vars in 912 bytes} [Tue Jun 7 22:03:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 22:03:03.445350 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221558e2-dd6b-4126-9a56-ef8a20ef609a None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:03.445635 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221558e2-dd6b-4126-9a56-ef8a20ef609a None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:03.445913 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221558e2-dd6b-4126-9a56-ef8a20ef609a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:03.491224 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-221558e2-dd6b-4126-9a56-ef8a20ef609a None None] MFA Rules not processed for user `dbe0303ea0764347bfb054870ebba41f`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:03.493268 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-221558e2-dd6b-4126-9a56-ef8a20ef609a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:03.539612 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 814/1638] 10.222.0.13 () {58 vars in 1009 bytes} [Tue Jun 7 22:03:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:03.541392 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8512d-a2a6-4f85-b220-cfee72744ba0 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:03.541625 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8512d-a2a6-4f85-b220-cfee72744ba0 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:03.541864 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8512d-a2a6-4f85-b220-cfee72744ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:03.609807 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-52b8512d-a2a6-4f85-b220-cfee72744ba0 None None] MFA Rules not processed for user `e78d9898f14c4b7987a9e7d3b4c3b164`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:03.612834 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-52b8512d-a2a6-4f85-b220-cfee72744ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:03.712736 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 825/1639] 10.222.0.13 () {58 vars in 1009 bytes} [Tue Jun 7 22:03:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:03.719900 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221558e2-dd6b-4126-9a56-ef8a20ef609a None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:03.720565 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 815/1640] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:03 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:03.742535 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8512d-a2a6-4f85-b220-cfee72744ba0 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:03.776693 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:03.815452 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8512d-a2a6-4f85-b220-cfee72744ba0 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:03.819462 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a728b65f-20bc-4ab5-8bfd-784f2ddefdc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:03.820194 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728b65f-20bc-4ab5-8bfd-784f2ddefdc6 service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:03.820546 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728b65f-20bc-4ab5-8bfd-784f2ddefdc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:03.820935 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728b65f-20bc-4ab5-8bfd-784f2ddefdc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:03.830675 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:03.848190 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a728b65f-20bc-4ab5-8bfd-784f2ddefdc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:03.851155 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a728b65f-20bc-4ab5-8bfd-784f2ddefdc6 service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:03.864807 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 816/1641] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:03:03.883131 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221558e2-dd6b-4126-9a56-ef8a20ef609a None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:03.888857 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2cfb3b-5f63-420c-bdc7-426b82660e4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:03.889272 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2cfb3b-5f63-420c-bdc7-426b82660e4c service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:03.889560 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2cfb3b-5f63-420c-bdc7-426b82660e4c service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:03.889894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2cfb3b-5f63-420c-bdc7-426b82660e4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:03.907402 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2cfb3b-5f63-420c-bdc7-426b82660e4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:03.911181 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2cfb3b-5f63-420c-bdc7-426b82660e4c service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:03.960659 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 826/1642] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:03:04.101115 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a728b65f-20bc-4ab5-8bfd-784f2ddefdc6 service cinder] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:04.112013 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a728b65f-20bc-4ab5-8bfd-784f2ddefdc6 service cinder] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:04.122930 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d8005c-7619-4332-be57-a1d183c0b9a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:04.123931 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d8005c-7619-4332-be57-a1d183c0b9a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:04.133334 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d8005c-7619-4332-be57-a1d183c0b9a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:04.133691 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d8005c-7619-4332-be57-a1d183c0b9a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:04.165075 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d8005c-7619-4332-be57-a1d183c0b9a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:04.168899 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d8005c-7619-4332-be57-a1d183c0b9a4 service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:04.197551 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 817/1643] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:04 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:03:04.582077 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2cfb3b-5f63-420c-bdc7-426b82660e4c service cinder] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:04.682817 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d8005c-7619-4332-be57-a1d183c0b9a4 service cinder] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:04.777697 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2cfb3b-5f63-420c-bdc7-426b82660e4c service cinder] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:04.795722 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9029f6d-abac-4aad-8271-165c89910340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:04.796825 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9029f6d-abac-4aad-8271-165c89910340 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:04.797114 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9029f6d-abac-4aad-8271-165c89910340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:04.797393 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9029f6d-abac-4aad-8271-165c89910340 admin admin] PATH_INFO: `/v3/users/9fe11b29cc0e4a78bddc435763a79adb` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:04.807695 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9029f6d-abac-4aad-8271-165c89910340 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fe11b29cc0e4a78bddc435763a79adb)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:04.815241 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9029f6d-abac-4aad-8271-165c89910340 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:04.904842 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d8005c-7619-4332-be57-a1d183c0b9a4 service cinder] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:04.922391 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6ef6ee-b81a-4dbb-a09e-0e5ee6cca753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:04.930987 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ef6ee-b81a-4dbb-a09e-0e5ee6cca753 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:04.931388 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ef6ee-b81a-4dbb-a09e-0e5ee6cca753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:04.931639 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ef6ee-b81a-4dbb-a09e-0e5ee6cca753 admin admin] PATH_INFO: `/v3/users/e174947bd6e640ae8169ccb80794abff` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:04.939449 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6ef6ee-b81a-4dbb-a09e-0e5ee6cca753 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e174947bd6e640ae8169ccb80794abff)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:04.941266 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: WARNING py.warnings [None req-7d6ef6ee-b81a-4dbb-a09e-0e5ee6cca753 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 22:03:04.941266 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: warnings.warn(msg) Jun 07 22:03:04.941266 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: Jun 07 22:03:04.942901 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6ef6ee-b81a-4dbb-a09e-0e5ee6cca753 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:04.970316 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c9029f6d-abac-4aad-8271-165c89910340 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fe11b29cc0e4a78bddc435763a79adb'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:04.996434 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c9029f6d-abac-4aad-8271-165c89910340 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fe11b29cc0e4a78bddc435763a79adb'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:05.040492 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c9029f6d-abac-4aad-8271-165c89910340 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fe11b29cc0e4a78bddc435763a79adb'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:05.041180 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c9029f6d-abac-4aad-8271-165c89910340 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fe11b29cc0e4a78bddc435763a79adb'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:05.043609 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c9029f6d-abac-4aad-8271-165c89910340 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fe11b29cc0e4a78bddc435763a79adb'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:05.070347 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 827/1644] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:03:04 2022] DELETE /identity/v3/users/9fe11b29cc0e4a78bddc435763a79adb => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:05.073884 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e38752-97cc-4c36-a60c-d4bef03a8383 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:05.074261 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e38752-97cc-4c36-a60c-d4bef03a8383 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:05.074530 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e38752-97cc-4c36-a60c-d4bef03a8383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:05.137456 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-7d6ef6ee-b81a-4dbb-a09e-0e5ee6cca753 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e174947bd6e640ae8169ccb80794abff'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:05.138194 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-7d6ef6ee-b81a-4dbb-a09e-0e5ee6cca753 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e174947bd6e640ae8169ccb80794abff'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:05.149061 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-d9e38752-97cc-4c36-a60c-d4bef03a8383 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:05.153209 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-7d6ef6ee-b81a-4dbb-a09e-0e5ee6cca753 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e174947bd6e640ae8169ccb80794abff'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:05.156233 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d9e38752-97cc-4c36-a60c-d4bef03a8383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:05.191737 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-7d6ef6ee-b81a-4dbb-a09e-0e5ee6cca753 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e174947bd6e640ae8169ccb80794abff'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:05.217772 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-7d6ef6ee-b81a-4dbb-a09e-0e5ee6cca753 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e174947bd6e640ae8169ccb80794abff'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:05.230895 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 818/1645] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:03:04 2022] DELETE /identity/v3/users/e174947bd6e640ae8169ccb80794abff => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:03:05.233162 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd2510-b0c1-4b06-9020-fba057df9562 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:05.233459 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd2510-b0c1-4b06-9020-fba057df9562 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:05.233740 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd2510-b0c1-4b06-9020-fba057df9562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:05.291306 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:05.298896 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-7dbd2510-b0c1-4b06-9020-fba057df9562 None None] MFA Rules not processed for user `1378a898f9534635857e5958d32f00c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:05.300243 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-7dbd2510-b0c1-4b06-9020-fba057df9562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:05.318536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 828/1646] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:05.318536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79be7cc1-ddce-4538-8d25-3c7ef8192fa5 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:05.318536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79be7cc1-ddce-4538-8d25-3c7ef8192fa5 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:05.318935 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79be7cc1-ddce-4538-8d25-3c7ef8192fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:05.341193 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-79be7cc1-ddce-4538-8d25-3c7ef8192fa5 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:05.343265 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-79be7cc1-ddce-4538-8d25-3c7ef8192fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:05.353725 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:05.464039 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 819/1647] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:05.465592 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbd2510-b0c1-4b06-9020-fba057df9562 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:05.474791 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-7dbd2510-b0c1-4b06-9020-fba057df9562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:05.484108 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 829/1648] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:05.484108 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79be7cc1-ddce-4538-8d25-3c7ef8192fa5 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:05.486023 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-79be7cc1-ddce-4538-8d25-3c7ef8192fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:05.505266 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79be7cc1-ddce-4538-8d25-3c7ef8192fa5 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:05.511965 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e771456-c2a7-41ce-a1cf-84f5b4522d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:05.513385 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e771456-c2a7-41ce-a1cf-84f5b4522d50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:05.513784 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e771456-c2a7-41ce-a1cf-84f5b4522d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:05.514172 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e771456-c2a7-41ce-a1cf-84f5b4522d50 admin admin] PATH_INFO: `/v3/users/9164a923819d4d34a679064e7f4b44d1` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:05.517773 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e771456-c2a7-41ce-a1cf-84f5b4522d50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9164a923819d4d34a679064e7f4b44d1)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:05.520669 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e771456-c2a7-41ce-a1cf-84f5b4522d50 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:05.599724 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbd2510-b0c1-4b06-9020-fba057df9562 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:05.608755 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd11abf-9aa4-4d11-b531-e06cad8005b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:05.609428 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd11abf-9aa4-4d11-b531-e06cad8005b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:05.609711 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd11abf-9aa4-4d11-b531-e06cad8005b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:05.609983 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd11abf-9aa4-4d11-b531-e06cad8005b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:05.616133 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3e771456-c2a7-41ce-a1cf-84f5b4522d50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9164a923819d4d34a679064e7f4b44d1'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:05.627664 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3e771456-c2a7-41ce-a1cf-84f5b4522d50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9164a923819d4d34a679064e7f4b44d1'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:05.629635 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd11abf-9aa4-4d11-b531-e06cad8005b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:05.631094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd11abf-9aa4-4d11-b531-e06cad8005b1 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:05.653160 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 820/1649] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:03:05.654510 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9854b5fe-83e9-4806-9c5f-76875c4c669e None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:05.654779 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9854b5fe-83e9-4806-9c5f-76875c4c669e None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:05.655059 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9854b5fe-83e9-4806-9c5f-76875c4c669e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:05.660654 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3e771456-c2a7-41ce-a1cf-84f5b4522d50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9164a923819d4d34a679064e7f4b44d1'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:05.662199 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3e771456-c2a7-41ce-a1cf-84f5b4522d50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9164a923819d4d34a679064e7f4b44d1'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:05.663215 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3e771456-c2a7-41ce-a1cf-84f5b4522d50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9164a923819d4d34a679064e7f4b44d1'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:05.693869 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 830/1650] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:03:05 2022] DELETE /identity/v3/users/9164a923819d4d34a679064e7f4b44d1 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:05.694242 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-9854b5fe-83e9-4806-9c5f-76875c4c669e None None] MFA Rules not processed for user `1378a898f9534635857e5958d32f00c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:05.695850 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e771456-c2a7-41ce-a1cf-84f5b4522d50 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:05.697338 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-9854b5fe-83e9-4806-9c5f-76875c4c669e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:05.703098 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-3e771456-c2a7-41ce-a1cf-84f5b4522d50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:05.800532 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 821/1651] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:05.802789 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e771456-c2a7-41ce-a1cf-84f5b4522d50 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:05.803862 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20efea2c-6281-4d8d-b3f7-f4cba4a5f967 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:05.804466 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20efea2c-6281-4d8d-b3f7-f4cba4a5f967 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:05.804900 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20efea2c-6281-4d8d-b3f7-f4cba4a5f967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:05.815242 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f631f1-c9ec-43e1-9d1a-b8247ae3f05c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:05.815894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f631f1-c9ec-43e1-9d1a-b8247ae3f05c service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:05.816175 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f631f1-c9ec-43e1-9d1a-b8247ae3f05c service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:05.816445 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f631f1-c9ec-43e1-9d1a-b8247ae3f05c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:05.818781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-23f631f1-c9ec-43e1-9d1a-b8247ae3f05c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:05.839601 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-20efea2c-6281-4d8d-b3f7-f4cba4a5f967 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:05.842579 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:05.845251 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-20efea2c-6281-4d8d-b3f7-f4cba4a5f967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:05.898673 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f631f1-c9ec-43e1-9d1a-b8247ae3f05c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:05.900225 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f631f1-c9ec-43e1-9d1a-b8247ae3f05c service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:05.916645 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 831/1652] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:03:05.917471 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f631f1-c9ec-43e1-9d1a-b8247ae3f05c service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:05.919502 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-23f631f1-c9ec-43e1-9d1a-b8247ae3f05c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:05.948641 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 822/1653] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:05.967260 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20efea2c-6281-4d8d-b3f7-f4cba4a5f967 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:05.969833 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-20efea2c-6281-4d8d-b3f7-f4cba4a5f967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:06.013356 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20efea2c-6281-4d8d-b3f7-f4cba4a5f967 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:06.025901 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c554a-20c0-413f-9eba-186f7b3801fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:06.028924 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c554a-20c0-413f-9eba-186f7b3801fb service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:06.029213 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c554a-20c0-413f-9eba-186f7b3801fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:06.029581 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c554a-20c0-413f-9eba-186f7b3801fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:06.063289 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6c554a-20c0-413f-9eba-186f7b3801fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:06.067526 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6c554a-20c0-413f-9eba-186f7b3801fb service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:06.082747 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:06.101720 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 823/1654] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:03:06.117946 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f631f1-c9ec-43e1-9d1a-b8247ae3f05c service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:06.128769 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df60815-717b-4927-a126-45a18d286a2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:06.129449 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df60815-717b-4927-a126-45a18d286a2d service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:06.129780 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df60815-717b-4927-a126-45a18d286a2d service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:06.130375 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df60815-717b-4927-a126-45a18d286a2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:06.132728 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-2df60815-717b-4927-a126-45a18d286a2d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:06.147806 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df60815-717b-4927-a126-45a18d286a2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:06.149556 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df60815-717b-4927-a126-45a18d286a2d service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:06.179357 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 832/1655] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:05 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:03:06.625221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c554a-20c0-413f-9eba-186f7b3801fb service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:06.628492 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-4a6c554a-20c0-413f-9eba-186f7b3801fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:06.649810 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c554a-20c0-413f-9eba-186f7b3801fb service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:06.656210 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff3d902-fc09-4cb9-8483-5716e93cf39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:06.657505 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff3d902-fc09-4cb9-8483-5716e93cf39c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:06.658376 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff3d902-fc09-4cb9-8483-5716e93cf39c admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:06.658553 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff3d902-fc09-4cb9-8483-5716e93cf39c admin admin] PATH_INFO: `/v3/projects/3ec97d2dae1f45ed82d63ba7a748cd7d` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:06.660666 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff3d902-fc09-4cb9-8483-5716e93cf39c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ec97d2dae1f45ed82d63ba7a748cd7d)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:06.662599 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff3d902-fc09-4cb9-8483-5716e93cf39c admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:06.798844 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-cff3d902-fc09-4cb9-8483-5716e93cf39c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ec97d2dae1f45ed82d63ba7a748cd7d', 'request_id': 'req-cff3d902-fc09-4cb9-8483-5716e93cf39c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:06.845426 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-cff3d902-fc09-4cb9-8483-5716e93cf39c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ec97d2dae1f45ed82d63ba7a748cd7d', 'request_id': 'req-cff3d902-fc09-4cb9-8483-5716e93cf39c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:06.869648 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-cff3d902-fc09-4cb9-8483-5716e93cf39c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ec97d2dae1f45ed82d63ba7a748cd7d', 'request_id': 'req-cff3d902-fc09-4cb9-8483-5716e93cf39c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:06.894030 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-cff3d902-fc09-4cb9-8483-5716e93cf39c admin admin] The token cache is being invalidate because project 3ec97d2dae1f45ed82d63ba7a748cd7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:03:06.894330 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-cff3d902-fc09-4cb9-8483-5716e93cf39c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:06.897579 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-cff3d902-fc09-4cb9-8483-5716e93cf39c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:06.901038 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 824/1656] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:03:06 2022] DELETE /identity/v3/projects/3ec97d2dae1f45ed82d63ba7a748cd7d => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:06.908039 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918ef660-f662-4209-8a75-741059787b89 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:06.908574 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918ef660-f662-4209-8a75-741059787b89 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:06.908574 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918ef660-f662-4209-8a75-741059787b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:06.930059 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-918ef660-f662-4209-8a75-741059787b89 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:06.931524 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-918ef660-f662-4209-8a75-741059787b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:07.025437 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 833/1657] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:07.026178 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a58eac-9ef9-45ca-b847-e4b7cb7795dc None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:07.026478 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a58eac-9ef9-45ca-b847-e4b7cb7795dc None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:07.026765 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a58eac-9ef9-45ca-b847-e4b7cb7795dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:07.056741 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-c9a58eac-9ef9-45ca-b847-e4b7cb7795dc None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:07.057993 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-c9a58eac-9ef9-45ca-b847-e4b7cb7795dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:07.072696 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 825/1658] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:07.076987 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918ef660-f662-4209-8a75-741059787b89 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:07.096435 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918ef660-f662-4209-8a75-741059787b89 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:07.102445 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12ecde-277d-4ea0-b721-7109af76da17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:07.103146 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12ecde-277d-4ea0-b721-7109af76da17 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:07.103487 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12ecde-277d-4ea0-b721-7109af76da17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:07.103772 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12ecde-277d-4ea0-b721-7109af76da17 admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:07.104870 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab12ecde-277d-4ea0-b721-7109af76da17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:03:07.105896 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab12ecde-277d-4ea0-b721-7109af76da17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:07.108900 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab12ecde-277d-4ea0-b721-7109af76da17 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:07.132690 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 834/1659] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:03:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:03:07.138202 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a58eac-9ef9-45ca-b847-e4b7cb7795dc None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:07.151192 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a58eac-9ef9-45ca-b847-e4b7cb7795dc None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:07.159221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c484eeb-c3c3-42e4-a1d1-f092068a4d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:07.160127 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c484eeb-c3c3-42e4-a1d1-f092068a4d48 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:07.160420 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c484eeb-c3c3-42e4-a1d1-f092068a4d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:07.160700 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c484eeb-c3c3-42e4-a1d1-f092068a4d48 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:07.161418 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c484eeb-c3c3-42e4-a1d1-f092068a4d48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:07.165506 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c484eeb-c3c3-42e4-a1d1-f092068a4d48 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:07.206935 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 826/1660] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:03:07 2022] POST /identity/v3/projects => generated 363 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:07.212418 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12ecde-277d-4ea0-b721-7109af76da17 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:07.234630 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12ecde-277d-4ea0-b721-7109af76da17 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:07.280542 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:07.355901 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923fa174-6143-4ae9-9031-e7c774fd64b6 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:07.356198 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923fa174-6143-4ae9-9031-e7c774fd64b6 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:07.356578 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923fa174-6143-4ae9-9031-e7c774fd64b6 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:07.360126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 827/1661] 10.222.0.13 () {54 vars in 912 bytes} [Tue Jun 7 22:03:07 2022] GET /identity => generated 270 bytes in 21 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 22:03:07.366226 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0902b-1bc7-4537-89de-337fb3feeea3 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:07.366226 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0902b-1bc7-4537-89de-337fb3feeea3 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:07.366226 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0902b-1bc7-4537-89de-337fb3feeea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:07.404888 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0844578-294f-4bcd-a7d7-c871334eec81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:07.414302 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0844578-294f-4bcd-a7d7-c871334eec81 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:07.416011 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0844578-294f-4bcd-a7d7-c871334eec81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:07.416320 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0844578-294f-4bcd-a7d7-c871334eec81 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:07.422124 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0844578-294f-4bcd-a7d7-c871334eec81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:07.431393 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0844578-294f-4bcd-a7d7-c871334eec81 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:07.500683 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 835/1662] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:03:07 2022] POST /identity/v3/users => generated 282 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:07.507968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0844578-294f-4bcd-a7d7-c871334eec81 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:07.514341 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-12f0902b-1bc7-4537-89de-337fb3feeea3 None None] MFA Rules not processed for user `10242251e6284217bfb79c67b1567f75`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:07.515295 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-e0844578-294f-4bcd-a7d7-c871334eec81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:07.515663 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-12f0902b-1bc7-4537-89de-337fb3feeea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:07.538437 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0844578-294f-4bcd-a7d7-c871334eec81 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:07.564197 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b52ee18-3fa5-4aac-91a3-749eb6c0e6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:07.568694 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52ee18-3fa5-4aac-91a3-749eb6c0e6a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:07.569434 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52ee18-3fa5-4aac-91a3-749eb6c0e6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:07.569895 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52ee18-3fa5-4aac-91a3-749eb6c0e6a5 admin admin] PATH_INFO: `/v3/projects/771bb741cb47444b8d7c56f9bcb62e93` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:07.573740 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b52ee18-3fa5-4aac-91a3-749eb6c0e6a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=771bb741cb47444b8d7c56f9bcb62e93)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:07.581134 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: WARNING py.warnings [None req-3b52ee18-3fa5-4aac-91a3-749eb6c0e6a5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 22:03:07.581134 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: warnings.warn(msg) Jun 07 22:03:07.581134 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: Jun 07 22:03:07.586461 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b52ee18-3fa5-4aac-91a3-749eb6c0e6a5 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:07.684324 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 828/1663] 10.222.0.13 () {58 vars in 1009 bytes} [Tue Jun 7 22:03:07 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:07.688261 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f0902b-1bc7-4537-89de-337fb3feeea3 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:07.726220 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f0902b-1bc7-4537-89de-337fb3feeea3 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:07.750632 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3b52ee18-3fa5-4aac-91a3-749eb6c0e6a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '771bb741cb47444b8d7c56f9bcb62e93', 'request_id': 'req-3b52ee18-3fa5-4aac-91a3-749eb6c0e6a5'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:07.767545 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3b52ee18-3fa5-4aac-91a3-749eb6c0e6a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '771bb741cb47444b8d7c56f9bcb62e93', 'request_id': 'req-3b52ee18-3fa5-4aac-91a3-749eb6c0e6a5'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:07.776318 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3b52ee18-3fa5-4aac-91a3-749eb6c0e6a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '771bb741cb47444b8d7c56f9bcb62e93', 'request_id': 'req-3b52ee18-3fa5-4aac-91a3-749eb6c0e6a5'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:07.818174 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3b52ee18-3fa5-4aac-91a3-749eb6c0e6a5 admin admin] The token cache is being invalidate because project 771bb741cb47444b8d7c56f9bcb62e93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:03:07.818472 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3b52ee18-3fa5-4aac-91a3-749eb6c0e6a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:07.820710 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3b52ee18-3fa5-4aac-91a3-749eb6c0e6a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:07.824823 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 836/1664] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:03:07 2022] DELETE /identity/v3/projects/771bb741cb47444b8d7c56f9bcb62e93 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:07.825844 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b52ee18-3fa5-4aac-91a3-749eb6c0e6a5 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:07.830253 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-3b52ee18-3fa5-4aac-91a3-749eb6c0e6a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:07.853627 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:07.880438 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68608da4-18de-4e92-b285-68c09f6c77ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:07.881475 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68608da4-18de-4e92-b285-68c09f6c77ba admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:07.882115 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68608da4-18de-4e92-b285-68c09f6c77ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:07.886853 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68608da4-18de-4e92-b285-68c09f6c77ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:07.888650 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68608da4-18de-4e92-b285-68c09f6c77ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:07.899554 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68608da4-18de-4e92-b285-68c09f6c77ba admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:07.942047 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 829/1665] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:07 2022] GET /identity/v3/roles => generated 1420 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:07.942475 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68608da4-18de-4e92-b285-68c09f6c77ba admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:07.947247 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-68608da4-18de-4e92-b285-68c09f6c77ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:07.978376 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68608da4-18de-4e92-b285-68c09f6c77ba admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:07.996419 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325fa4fb-f3f9-4480-9586-c5a59d7d5efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:07.999348 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325fa4fb-f3f9-4480-9586-c5a59d7d5efb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:07.999666 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325fa4fb-f3f9-4480-9586-c5a59d7d5efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:07.999950 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325fa4fb-f3f9-4480-9586-c5a59d7d5efb admin admin] PATH_INFO: `/v3/users/b8ad47be8834455c9e0a70f453f8fbec` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:08.006469 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325fa4fb-f3f9-4480-9586-c5a59d7d5efb admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8ad47be8834455c9e0a70f453f8fbec)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:08.008541 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325fa4fb-f3f9-4480-9586-c5a59d7d5efb admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:08.031530 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b52ee18-3fa5-4aac-91a3-749eb6c0e6a5 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:08.041167 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bf38cb-d2cf-4152-bbc8-b49dcb165647 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:08.041837 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bf38cb-d2cf-4152-bbc8-b49dcb165647 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:08.042125 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bf38cb-d2cf-4152-bbc8-b49dcb165647 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:08.042406 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bf38cb-d2cf-4152-bbc8-b49dcb165647 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:08.056733 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-b1bf38cb-d2cf-4152-bbc8-b49dcb165647 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:08.102696 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bf38cb-d2cf-4152-bbc8-b49dcb165647 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:08.104779 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bf38cb-d2cf-4152-bbc8-b49dcb165647 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:08.130821 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 837/1666] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:03:08.133849 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bf38cb-d2cf-4152-bbc8-b49dcb165647 service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:08.138290 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-b1bf38cb-d2cf-4152-bbc8-b49dcb165647 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:08.153121 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-325fa4fb-f3f9-4480-9586-c5a59d7d5efb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8ad47be8834455c9e0a70f453f8fbec'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:08.155057 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-325fa4fb-f3f9-4480-9586-c5a59d7d5efb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8ad47be8834455c9e0a70f453f8fbec'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:08.174510 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-325fa4fb-f3f9-4480-9586-c5a59d7d5efb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8ad47be8834455c9e0a70f453f8fbec'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:08.192822 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-325fa4fb-f3f9-4480-9586-c5a59d7d5efb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8ad47be8834455c9e0a70f453f8fbec'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:08.204679 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-325fa4fb-f3f9-4480-9586-c5a59d7d5efb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8ad47be8834455c9e0a70f453f8fbec'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:08.207711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bf38cb-d2cf-4152-bbc8-b49dcb165647 service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:08.217463 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 830/1667] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:03:07 2022] DELETE /identity/v3/users/b8ad47be8834455c9e0a70f453f8fbec => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:08.223169 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e574789-4954-4ef1-9ec0-c8989bca1837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:08.225380 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e574789-4954-4ef1-9ec0-c8989bca1837 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:08.226144 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e574789-4954-4ef1-9ec0-c8989bca1837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:08.226732 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e574789-4954-4ef1-9ec0-c8989bca1837 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:08.227788 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e574789-4954-4ef1-9ec0-c8989bca1837 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:08.232449 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e574789-4954-4ef1-9ec0-c8989bca1837 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:08.247701 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 838/1668] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:08 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:08.251601 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325fa4fb-f3f9-4480-9586-c5a59d7d5efb admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:08.253173 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-325fa4fb-f3f9-4480-9586-c5a59d7d5efb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:08.387603 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325fa4fb-f3f9-4480-9586-c5a59d7d5efb admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:08.398781 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c369554-2626-4a5a-ad9c-ccb038e99aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:08.400247 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c369554-2626-4a5a-ad9c-ccb038e99aa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:08.401078 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c369554-2626-4a5a-ad9c-ccb038e99aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:08.401078 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c369554-2626-4a5a-ad9c-ccb038e99aa8 admin admin] PATH_INFO: `/v3/projects/4b12be8645a94204936370294de6c1cb/users/93604b93b7df47aba313bebca0c99fb1/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:08.450145 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c369554-2626-4a5a-ad9c-ccb038e99aa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b12be8645a94204936370294de6c1cb, user_id=93604b93b7df47aba313bebca0c99fb1, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:08.453786 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c369554-2626-4a5a-ad9c-ccb038e99aa8 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:08.474487 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 831/1669] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:03:08 2022] PUT /identity/v3/projects/4b12be8645a94204936370294de6c1cb/users/93604b93b7df47aba313bebca0c99fb1/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:08.487818 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0d46b-3ff1-4727-b9e5-81f6d404d071 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:08.488265 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0d46b-3ff1-4727-b9e5-81f6d404d071 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:08.488767 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0d46b-3ff1-4727-b9e5-81f6d404d071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:08.562495 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-3ba0d46b-3ff1-4727-b9e5-81f6d404d071 None None] MFA Rules not processed for user `93604b93b7df47aba313bebca0c99fb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:08.564312 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-3ba0d46b-3ff1-4727-b9e5-81f6d404d071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:08.621682 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:08.787866 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 839/1670] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:08 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:08.792199 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c369554-2626-4a5a-ad9c-ccb038e99aa8 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:08.950160 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:09.014181 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba0d46b-3ff1-4727-b9e5-81f6d404d071 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:09.016378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-3ba0d46b-3ff1-4727-b9e5-81f6d404d071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:09.041859 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c369554-2626-4a5a-ad9c-ccb038e99aa8 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:09.051357 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba0d46b-3ff1-4727-b9e5-81f6d404d071 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:09.064156 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a889165-9737-41ca-8587-b46a95b4ffc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:09.064860 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a889165-9737-41ca-8587-b46a95b4ffc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:09.065151 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a889165-9737-41ca-8587-b46a95b4ffc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:09.065433 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a889165-9737-41ca-8587-b46a95b4ffc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:09.068962 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc723e6-876a-4b2e-ada7-b3334b4269b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:09.069860 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc723e6-876a-4b2e-ada7-b3334b4269b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:09.070140 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc723e6-876a-4b2e-ada7-b3334b4269b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:09.070419 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc723e6-876a-4b2e-ada7-b3334b4269b4 admin admin] PATH_INFO: `/v3/projects/0a9c7772d09c4827a38b523447736134` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:09.071634 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a889165-9737-41ca-8587-b46a95b4ffc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:09.073587 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a889165-9737-41ca-8587-b46a95b4ffc5 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:09.079751 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc723e6-876a-4b2e-ada7-b3334b4269b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a9c7772d09c4827a38b523447736134)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:09.085734 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc723e6-876a-4b2e-ada7-b3334b4269b4 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:09.162858 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 832/1671] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:03:08 2022] POST /identity/v3/projects => generated 365 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:03:09.163781 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a889165-9737-41ca-8587-b46a95b4ffc5 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:09.169664 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-9a889165-9737-41ca-8587-b46a95b4ffc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:09.247348 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3fc723e6-876a-4b2e-ada7-b3334b4269b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a9c7772d09c4827a38b523447736134', 'request_id': 'req-3fc723e6-876a-4b2e-ada7-b3334b4269b4'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:09.267197 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3fc723e6-876a-4b2e-ada7-b3334b4269b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a9c7772d09c4827a38b523447736134', 'request_id': 'req-3fc723e6-876a-4b2e-ada7-b3334b4269b4'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:09.281918 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3fc723e6-876a-4b2e-ada7-b3334b4269b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a9c7772d09c4827a38b523447736134', 'request_id': 'req-3fc723e6-876a-4b2e-ada7-b3334b4269b4'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:09.324430 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3fc723e6-876a-4b2e-ada7-b3334b4269b4 admin admin] The token cache is being invalidate because project 0a9c7772d09c4827a38b523447736134 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:03:09.324926 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3fc723e6-876a-4b2e-ada7-b3334b4269b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:09.325802 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3fc723e6-876a-4b2e-ada7-b3334b4269b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:09.331528 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 840/1672] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:03:09 2022] DELETE /identity/v3/projects/0a9c7772d09c4827a38b523447736134 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:03:09.335151 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc723e6-876a-4b2e-ada7-b3334b4269b4 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:09.337480 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-3fc723e6-876a-4b2e-ada7-b3334b4269b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:09.341820 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a889165-9737-41ca-8587-b46a95b4ffc5 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:09.540533 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f44885-4ed7-4e13-9f23-6642e34c0bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:09.541704 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f44885-4ed7-4e13-9f23-6642e34c0bd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:09.542478 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f44885-4ed7-4e13-9f23-6642e34c0bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:09.543489 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f44885-4ed7-4e13-9f23-6642e34c0bd3 admin admin] PATH_INFO: `/v3/users/17ce5e31e7724b77ab47c52ee0d3021e` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:09.546028 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f44885-4ed7-4e13-9f23-6642e34c0bd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17ce5e31e7724b77ab47c52ee0d3021e)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:09.549753 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f44885-4ed7-4e13-9f23-6642e34c0bd3 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:09.562099 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc723e6-876a-4b2e-ada7-b3334b4269b4 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:09.573929 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98e4076-73ef-4ac9-999d-2a137ea1e15d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:09.574644 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e4076-73ef-4ac9-999d-2a137ea1e15d admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:09.574931 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e4076-73ef-4ac9-999d-2a137ea1e15d admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:09.575213 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e4076-73ef-4ac9-999d-2a137ea1e15d admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:09.578719 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98e4076-73ef-4ac9-999d-2a137ea1e15d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:09.584211 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98e4076-73ef-4ac9-999d-2a137ea1e15d admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:09.644275 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 841/1673] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:03:09 2022] POST /identity/v3/users => generated 289 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:09.647449 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98e4076-73ef-4ac9-999d-2a137ea1e15d admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:09.653085 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-f98e4076-73ef-4ac9-999d-2a137ea1e15d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:09.674724 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98e4076-73ef-4ac9-999d-2a137ea1e15d admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:09.692476 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a964b4-3189-41b8-bec7-51733f4ed9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:09.693559 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a964b4-3189-41b8-bec7-51733f4ed9a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:09.696537 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a964b4-3189-41b8-bec7-51733f4ed9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:09.696847 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a964b4-3189-41b8-bec7-51733f4ed9a1 admin admin] PATH_INFO: `/v3/projects/0cd1fa7e85f54b228b5015a172ffb10e` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:09.698251 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a964b4-3189-41b8-bec7-51733f4ed9a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cd1fa7e85f54b228b5015a172ffb10e)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:09.701798 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a964b4-3189-41b8-bec7-51733f4ed9a1 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:09.702261 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-76f44885-4ed7-4e13-9f23-6642e34c0bd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17ce5e31e7724b77ab47c52ee0d3021e'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:09.703033 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-76f44885-4ed7-4e13-9f23-6642e34c0bd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17ce5e31e7724b77ab47c52ee0d3021e'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:09.719776 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-76f44885-4ed7-4e13-9f23-6642e34c0bd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17ce5e31e7724b77ab47c52ee0d3021e'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:09.764931 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-76f44885-4ed7-4e13-9f23-6642e34c0bd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17ce5e31e7724b77ab47c52ee0d3021e'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:09.769224 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:09.774990 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:09.780079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-76f44885-4ed7-4e13-9f23-6642e34c0bd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17ce5e31e7724b77ab47c52ee0d3021e'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:09.790251 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 833/1674] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:03:09 2022] DELETE /identity/v3/users/17ce5e31e7724b77ab47c52ee0d3021e => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:09.795374 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f44885-4ed7-4e13-9f23-6642e34c0bd3 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:09.798754 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-76f44885-4ed7-4e13-9f23-6642e34c0bd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:09.812141 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-88a964b4-3189-41b8-bec7-51733f4ed9a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cd1fa7e85f54b228b5015a172ffb10e', 'request_id': 'req-88a964b4-3189-41b8-bec7-51733f4ed9a1'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:09.828452 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-88a964b4-3189-41b8-bec7-51733f4ed9a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cd1fa7e85f54b228b5015a172ffb10e', 'request_id': 'req-88a964b4-3189-41b8-bec7-51733f4ed9a1'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:09.844899 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-88a964b4-3189-41b8-bec7-51733f4ed9a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cd1fa7e85f54b228b5015a172ffb10e', 'request_id': 'req-88a964b4-3189-41b8-bec7-51733f4ed9a1'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:09.876815 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-88a964b4-3189-41b8-bec7-51733f4ed9a1 admin admin] The token cache is being invalidate because project 0cd1fa7e85f54b228b5015a172ffb10e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:03:09.877702 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-88a964b4-3189-41b8-bec7-51733f4ed9a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:09.878911 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-88a964b4-3189-41b8-bec7-51733f4ed9a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:09.883929 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 842/1675] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:03:09 2022] DELETE /identity/v3/projects/0cd1fa7e85f54b228b5015a172ffb10e => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:09.886859 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a964b4-3189-41b8-bec7-51733f4ed9a1 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:09.892085 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-88a964b4-3189-41b8-bec7-51733f4ed9a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:09.973603 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f44885-4ed7-4e13-9f23-6642e34c0bd3 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:09.981167 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba139ed9-c2e3-4401-9061-6297e03ca0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:09.982817 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba139ed9-c2e3-4401-9061-6297e03ca0a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:09.983499 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba139ed9-c2e3-4401-9061-6297e03ca0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:09.984073 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba139ed9-c2e3-4401-9061-6297e03ca0a3 admin admin] PATH_INFO: `/v3/users/e78d9898f14c4b7987a9e7d3b4c3b164` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:09.985819 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba139ed9-c2e3-4401-9061-6297e03ca0a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e78d9898f14c4b7987a9e7d3b4c3b164)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:09.988850 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba139ed9-c2e3-4401-9061-6297e03ca0a3 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:10.066300 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a964b4-3189-41b8-bec7-51733f4ed9a1 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:10.069144 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-ba139ed9-c2e3-4401-9061-6297e03ca0a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e78d9898f14c4b7987a9e7d3b4c3b164'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:10.070058 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-ba139ed9-c2e3-4401-9061-6297e03ca0a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e78d9898f14c4b7987a9e7d3b4c3b164'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:10.082849 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa588ea-68f3-456a-87fa-1cd025ea5af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:10.083641 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-ba139ed9-c2e3-4401-9061-6297e03ca0a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e78d9898f14c4b7987a9e7d3b4c3b164'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:10.083871 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa588ea-68f3-456a-87fa-1cd025ea5af6 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:10.084259 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa588ea-68f3-456a-87fa-1cd025ea5af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:10.084632 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa588ea-68f3-456a-87fa-1cd025ea5af6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:10.085483 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa588ea-68f3-456a-87fa-1cd025ea5af6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:10.090224 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa588ea-68f3-456a-87fa-1cd025ea5af6 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:10.116699 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 843/1676] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:09 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:10.116699 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645ea5c-d5a8-4507-b1d0-1ca9183d8e64 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:10.116699 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645ea5c-d5a8-4507-b1d0-1ca9183d8e64 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:10.118352 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-ba139ed9-c2e3-4401-9061-6297e03ca0a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e78d9898f14c4b7987a9e7d3b4c3b164'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:10.121796 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645ea5c-d5a8-4507-b1d0-1ca9183d8e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:10.138434 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-ba139ed9-c2e3-4401-9061-6297e03ca0a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e78d9898f14c4b7987a9e7d3b4c3b164'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:10.152088 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 834/1677] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:03:09 2022] DELETE /identity/v3/users/e78d9898f14c4b7987a9e7d3b4c3b164 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:10.154337 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c57977-96ab-4dac-96cc-4b130aa169a3 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:10.154787 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c57977-96ab-4dac-96cc-4b130aa169a3 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:10.156398 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c57977-96ab-4dac-96cc-4b130aa169a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:10.164139 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-f645ea5c-d5a8-4507-b1d0-1ca9183d8e64 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:10.165656 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-f645ea5c-d5a8-4507-b1d0-1ca9183d8e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:10.194895 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-93c57977-96ab-4dac-96cc-4b130aa169a3 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:10.196255 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-93c57977-96ab-4dac-96cc-4b130aa169a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:10.318803 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 844/1678] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:10.326195 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f645ea5c-d5a8-4507-b1d0-1ca9183d8e64 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:10.334516 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-f645ea5c-d5a8-4507-b1d0-1ca9183d8e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:10.359959 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f645ea5c-d5a8-4507-b1d0-1ca9183d8e64 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:10.377323 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fef2b8-41e1-4613-87ce-df5258c6cfa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:10.378535 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fef2b8-41e1-4613-87ce-df5258c6cfa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:10.379016 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fef2b8-41e1-4613-87ce-df5258c6cfa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:10.379418 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fef2b8-41e1-4613-87ce-df5258c6cfa7 admin admin] PATH_INFO: `/v3/projects/06aee994b46e45ed9ff3d56bfbf49326/users/65718ab4d9be40fa83ef31498bc015fa/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:10.392085 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 835/1679] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:10.393188 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c57977-96ab-4dac-96cc-4b130aa169a3 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:10.397981 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-93c57977-96ab-4dac-96cc-4b130aa169a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:10.417402 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fef2b8-41e1-4613-87ce-df5258c6cfa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06aee994b46e45ed9ff3d56bfbf49326, user_id=65718ab4d9be40fa83ef31498bc015fa, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:10.423295 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fef2b8-41e1-4613-87ce-df5258c6cfa7 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:10.437399 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:10.458920 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 845/1680] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:03:10 2022] PUT /identity/v3/projects/06aee994b46e45ed9ff3d56bfbf49326/users/65718ab4d9be40fa83ef31498bc015fa/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:10.462775 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802801c3-9957-4090-9e45-dea819efcc9b None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:10.466802 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802801c3-9957-4090-9e45-dea819efcc9b None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:10.467173 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802801c3-9957-4090-9e45-dea819efcc9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:10.515219 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-802801c3-9957-4090-9e45-dea819efcc9b None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:10.516704 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-802801c3-9957-4090-9e45-dea819efcc9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:10.560984 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c57977-96ab-4dac-96cc-4b130aa169a3 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:10.569506 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:10.656651 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 846/1681] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:10.657678 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802801c3-9957-4090-9e45-dea819efcc9b None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:10.689185 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802801c3-9957-4090-9e45-dea819efcc9b None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:10.701911 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab29e8c-f30e-4263-96dc-08eb1ac7a9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:10.703010 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab29e8c-f30e-4263-96dc-08eb1ac7a9fa admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:10.703527 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab29e8c-f30e-4263-96dc-08eb1ac7a9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:10.703996 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab29e8c-f30e-4263-96dc-08eb1ac7a9fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:10.704955 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab29e8c-f30e-4263-96dc-08eb1ac7a9fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:10.708323 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab29e8c-f30e-4263-96dc-08eb1ac7a9fa admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:10.734883 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 847/1682] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:10.737709 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab29e8c-f30e-4263-96dc-08eb1ac7a9fa admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:10.758976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab29e8c-f30e-4263-96dc-08eb1ac7a9fa admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:10.773765 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e5e63-44ee-4de0-822c-0ddc1ee041a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:10.774609 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e5e63-44ee-4de0-822c-0ddc1ee041a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:10.776160 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e5e63-44ee-4de0-822c-0ddc1ee041a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:10.776568 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e5e63-44ee-4de0-822c-0ddc1ee041a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:10.784699 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6e5e63-44ee-4de0-822c-0ddc1ee041a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:03:10.785167 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6e5e63-44ee-4de0-822c-0ddc1ee041a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:10.786535 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c561b781-3e48-4847-969e-c95cfaa38cf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:10.789177 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6e5e63-44ee-4de0-822c-0ddc1ee041a5 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:10.789900 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c561b781-3e48-4847-969e-c95cfaa38cf3 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:10.790195 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c561b781-3e48-4847-969e-c95cfaa38cf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:10.790478 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c561b781-3e48-4847-969e-c95cfaa38cf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:10.828809 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 848/1683] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:03:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:03:10.829902 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e5e63-44ee-4de0-822c-0ddc1ee041a5 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:10.849255 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c561b781-3e48-4847-969e-c95cfaa38cf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:10.850870 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e5e63-44ee-4de0-822c-0ddc1ee041a5 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:10.857245 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c561b781-3e48-4847-969e-c95cfaa38cf3 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:10.863582 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bcd9c5-363c-4626-a274-af028cefb430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:10.864617 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcd9c5-363c-4626-a274-af028cefb430 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:10.866709 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcd9c5-363c-4626-a274-af028cefb430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:10.867897 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcd9c5-363c-4626-a274-af028cefb430 admin admin] PATH_INFO: `/v3/projects/06aee994b46e45ed9ff3d56bfbf49326/users/65718ab4d9be40fa83ef31498bc015fa/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:10.882508 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bcd9c5-363c-4626-a274-af028cefb430 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06aee994b46e45ed9ff3d56bfbf49326, user_id=65718ab4d9be40fa83ef31498bc015fa, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:10.882780 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 836/1684] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:03:10.884073 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c561b781-3e48-4847-969e-c95cfaa38cf3 service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:10.892298 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bcd9c5-363c-4626-a274-af028cefb430 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:10.894565 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c561b781-3e48-4847-969e-c95cfaa38cf3 service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:10.899913 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7ae7c5-7f4c-4f7c-9db0-23aa2cb8a76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:10.901098 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ae7c5-7f4c-4f7c-9db0-23aa2cb8a76c admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:10.901098 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ae7c5-7f4c-4f7c-9db0-23aa2cb8a76c admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:10.901263 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ae7c5-7f4c-4f7c-9db0-23aa2cb8a76c admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:10.902534 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7ae7c5-7f4c-4f7c-9db0-23aa2cb8a76c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:10.905000 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7ae7c5-7f4c-4f7c-9db0-23aa2cb8a76c admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:10.933361 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 849/1685] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:03:10 2022] PUT /identity/v3/projects/06aee994b46e45ed9ff3d56bfbf49326/users/65718ab4d9be40fa83ef31498bc015fa/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:10.938904 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857b014-b812-459d-8497-eb4b4317edca None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:10.939290 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857b014-b812-459d-8497-eb4b4317edca None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:10.939729 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857b014-b812-459d-8497-eb4b4317edca None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:10.941227 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 850/1686] 10.222.0.13 () {54 vars in 960 bytes} [Tue Jun 7 22:03:10 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 22:03:10.943558 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac79739-933a-4aa9-aacc-764ea6a4c1ec None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:10.943927 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac79739-933a-4aa9-aacc-764ea6a4c1ec None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:10.944698 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac79739-933a-4aa9-aacc-764ea6a4c1ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:10.969576 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 837/1687] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:03:10 2022] POST /identity/v3/projects => generated 379 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:10.987278 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b645d58a-545e-44f8-bba7-b03732467edf None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:10.988002 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b645d58a-545e-44f8-bba7-b03732467edf None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:10.988002 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b645d58a-545e-44f8-bba7-b03732467edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:11.013085 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-dac79739-933a-4aa9-aacc-764ea6a4c1ec None None] MFA Rules not processed for user `65718ab4d9be40fa83ef31498bc015fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:11.015356 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-dac79739-933a-4aa9-aacc-764ea6a4c1ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:11.052419 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-b645d58a-545e-44f8-bba7-b03732467edf None None] MFA Rules not processed for user `02941c6ad2d54e48b1114d9e030c66c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:11.053652 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-b645d58a-545e-44f8-bba7-b03732467edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:11.234051 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 838/1688] 10.222.0.13 () {58 vars in 1057 bytes} [Tue Jun 7 22:03:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:11.236096 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b645d58a-545e-44f8-bba7-b03732467edf None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:11.308432 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 851/1689] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:10 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:11.311020 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac79739-933a-4aa9-aacc-764ea6a4c1ec None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:11.314464 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-dac79739-933a-4aa9-aacc-764ea6a4c1ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:11.369975 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:11.444136 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b645d58a-545e-44f8-bba7-b03732467edf None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:11.454366 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464212b6-ccd0-490f-94a9-9cfe34bd0206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:11.455413 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464212b6-ccd0-490f-94a9-9cfe34bd0206 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:11.456313 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464212b6-ccd0-490f-94a9-9cfe34bd0206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:11.457728 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464212b6-ccd0-490f-94a9-9cfe34bd0206 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:11.459607 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464212b6-ccd0-490f-94a9-9cfe34bd0206 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:11.465122 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464212b6-ccd0-490f-94a9-9cfe34bd0206 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:11.513757 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 839/1690] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:03:11 2022] POST /identity/v3/users => generated 290 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:11.515423 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464212b6-ccd0-490f-94a9-9cfe34bd0206 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:11.544174 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464212b6-ccd0-490f-94a9-9cfe34bd0206 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:11.551130 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac79739-933a-4aa9-aacc-764ea6a4c1ec None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:11.561888 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034af0bd-5089-42cf-b315-288cfd51fcd3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02941c6ad2d54e48b1114d9e030c66c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:11.563916 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034af0bd-5089-42cf-b315-288cfd51fcd3 service placement] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:11.566551 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034af0bd-5089-42cf-b315-288cfd51fcd3 service placement] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:11.566983 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034af0bd-5089-42cf-b315-288cfd51fcd3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:11.570423 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-034af0bd-5089-42cf-b315-288cfd51fcd3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:11.577751 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1695ad3-8d14-4e99-8f67-49b75a3050b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:11.578655 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1695ad3-8d14-4e99-8f67-49b75a3050b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:11.578941 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1695ad3-8d14-4e99-8f67-49b75a3050b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:11.579500 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1695ad3-8d14-4e99-8f67-49b75a3050b6 admin admin] PATH_INFO: `/v3/projects/c9c5640ce3ba4ab5aa7029802fb23cce` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:11.581904 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1695ad3-8d14-4e99-8f67-49b75a3050b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9c5640ce3ba4ab5aa7029802fb23cce)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:11.583920 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1695ad3-8d14-4e99-8f67-49b75a3050b6 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:11.601727 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034af0bd-5089-42cf-b315-288cfd51fcd3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:11.606486 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034af0bd-5089-42cf-b315-288cfd51fcd3 service placement] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:11.633133 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 840/1691] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:03:11.635913 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034af0bd-5089-42cf-b315-288cfd51fcd3 service placement] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:11.640048 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-034af0bd-5089-42cf-b315-288cfd51fcd3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:11.654272 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:11.717098 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-b1695ad3-8d14-4e99-8f67-49b75a3050b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9c5640ce3ba4ab5aa7029802fb23cce', 'request_id': 'req-b1695ad3-8d14-4e99-8f67-49b75a3050b6'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:11.727696 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-b1695ad3-8d14-4e99-8f67-49b75a3050b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9c5640ce3ba4ab5aa7029802fb23cce', 'request_id': 'req-b1695ad3-8d14-4e99-8f67-49b75a3050b6'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:11.736897 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-b1695ad3-8d14-4e99-8f67-49b75a3050b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9c5640ce3ba4ab5aa7029802fb23cce', 'request_id': 'req-b1695ad3-8d14-4e99-8f67-49b75a3050b6'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:11.788831 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-b1695ad3-8d14-4e99-8f67-49b75a3050b6 admin admin] The token cache is being invalidate because project c9c5640ce3ba4ab5aa7029802fb23cce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:03:11.789136 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-b1695ad3-8d14-4e99-8f67-49b75a3050b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:11.789870 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-b1695ad3-8d14-4e99-8f67-49b75a3050b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:11.791514 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 852/1692] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:03:11 2022] DELETE /identity/v3/projects/c9c5640ce3ba4ab5aa7029802fb23cce => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:11.792309 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1695ad3-8d14-4e99-8f67-49b75a3050b6 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:11.796065 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034af0bd-5089-42cf-b315-288cfd51fcd3 service placement] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:11.809002 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-b1695ad3-8d14-4e99-8f67-49b75a3050b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:11.938953 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2380d7-c2c5-4586-b4eb-ac7e5dbe4e01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:11.939676 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2380d7-c2c5-4586-b4eb-ac7e5dbe4e01 service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:11.939964 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2380d7-c2c5-4586-b4eb-ac7e5dbe4e01 service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:11.940243 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2380d7-c2c5-4586-b4eb-ac7e5dbe4e01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:11.947995 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1695ad3-8d14-4e99-8f67-49b75a3050b6 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:11.948380 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-ea2380d7-c2c5-4586-b4eb-ac7e5dbe4e01 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:11.958727 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475f78b5-238f-488b-8eb1-dd236685b96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:11.959461 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f78b5-238f-488b-8eb1-dd236685b96f admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:11.959805 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f78b5-238f-488b-8eb1-dd236685b96f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:11.960089 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f78b5-238f-488b-8eb1-dd236685b96f admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:11.961729 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475f78b5-238f-488b-8eb1-dd236685b96f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:11.967070 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475f78b5-238f-488b-8eb1-dd236685b96f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:11.990602 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 853/1693] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:11 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:11.993016 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cae5ac-6203-4384-9120-a5ef2d49977f None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:11.993304 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cae5ac-6203-4384-9120-a5ef2d49977f None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:11.993579 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cae5ac-6203-4384-9120-a5ef2d49977f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:12.035246 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-c1cae5ac-6203-4384-9120-a5ef2d49977f None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:12.038886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c1cae5ac-6203-4384-9120-a5ef2d49977f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:12.060652 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 854/1694] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:12.061680 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cae5ac-6203-4384-9120-a5ef2d49977f None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:12.084577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cae5ac-6203-4384-9120-a5ef2d49977f None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:12.098074 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94101afb-9fb7-4545-9a74-d6b713480888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:12.098847 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94101afb-9fb7-4545-9a74-d6b713480888 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:12.099168 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94101afb-9fb7-4545-9a74-d6b713480888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:12.099499 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94101afb-9fb7-4545-9a74-d6b713480888 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:12.100310 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94101afb-9fb7-4545-9a74-d6b713480888 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:12.104591 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94101afb-9fb7-4545-9a74-d6b713480888 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:12.129516 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 855/1695] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:12 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:12.130927 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2380d7-c2c5-4586-b4eb-ac7e5dbe4e01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:12.133764 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2c35f-2327-4e7f-9870-cbb50cc69c96 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:12.134165 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2c35f-2327-4e7f-9870-cbb50cc69c96 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:12.134614 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2c35f-2327-4e7f-9870-cbb50cc69c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:12.135005 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2380d7-c2c5-4586-b4eb-ac7e5dbe4e01 service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:12.152861 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 841/1696] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:11 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:03:12.154469 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2380d7-c2c5-4586-b4eb-ac7e5dbe4e01 service cinder] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:12.176439 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-ebe2c35f-2327-4e7f-9870-cbb50cc69c96 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:12.177912 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-ebe2c35f-2327-4e7f-9870-cbb50cc69c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:12.200197 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 856/1697] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:12.204537 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe2c35f-2327-4e7f-9870-cbb50cc69c96 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:12.208686 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2380d7-c2c5-4586-b4eb-ac7e5dbe4e01 service cinder] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:12.224036 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe2c35f-2327-4e7f-9870-cbb50cc69c96 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:12.225511 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2378cf3d-9f09-458b-bc92-2c7ef2b121e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:12.228480 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2378cf3d-9f09-458b-bc92-2c7ef2b121e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:12.229758 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2378cf3d-9f09-458b-bc92-2c7ef2b121e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:12.230301 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2378cf3d-9f09-458b-bc92-2c7ef2b121e6 admin admin] PATH_INFO: `/v3/projects/a2d5c5b32045404486f39af9c2598716/users/c48b588bf82b4eb9ae2b6d7f0b137e3f/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:12.239052 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e231f101-dee4-4d0b-90a4-08e36c4a20f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:12.241545 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231f101-dee4-4d0b-90a4-08e36c4a20f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:12.241858 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231f101-dee4-4d0b-90a4-08e36c4a20f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:12.242437 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231f101-dee4-4d0b-90a4-08e36c4a20f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:12.243605 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e231f101-dee4-4d0b-90a4-08e36c4a20f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:03:12.243934 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e231f101-dee4-4d0b-90a4-08e36c4a20f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:12.247665 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e231f101-dee4-4d0b-90a4-08e36c4a20f1 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:12.277425 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2378cf3d-9f09-458b-bc92-2c7ef2b121e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2d5c5b32045404486f39af9c2598716, user_id=c48b588bf82b4eb9ae2b6d7f0b137e3f, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:12.277999 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 857/1698] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:03:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:03:12.282370 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e231f101-dee4-4d0b-90a4-08e36c4a20f1 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:12.294607 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2378cf3d-9f09-458b-bc92-2c7ef2b121e6 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:12.305109 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e231f101-dee4-4d0b-90a4-08e36c4a20f1 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:12.324688 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ef6ec3-4dd3-405b-94e1-6ecb0594d9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:12.327584 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef6ec3-4dd3-405b-94e1-6ecb0594d9de admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:12.327876 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef6ec3-4dd3-405b-94e1-6ecb0594d9de admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:12.328158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef6ec3-4dd3-405b-94e1-6ecb0594d9de admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:12.331248 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ef6ec3-4dd3-405b-94e1-6ecb0594d9de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:12.336752 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ef6ec3-4dd3-405b-94e1-6ecb0594d9de admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:12.338259 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 842/1699] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:03:12 2022] PUT /identity/v3/projects/a2d5c5b32045404486f39af9c2598716/users/c48b588bf82b4eb9ae2b6d7f0b137e3f/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:12.339912 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2378cf3d-9f09-458b-bc92-2c7ef2b121e6 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:12.352913 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-2378cf3d-9f09-458b-bc92-2c7ef2b121e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:12.369942 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:12.427890 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 858/1700] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:03:12 2022] POST /identity/v3/projects => generated 377 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:12.428908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418eddd-5190-4e01-b3d1-dc991dc3139c None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:12.431597 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418eddd-5190-4e01-b3d1-dc991dc3139c None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:12.431894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418eddd-5190-4e01-b3d1-dc991dc3139c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:12.482756 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-b418eddd-5190-4e01-b3d1-dc991dc3139c None None] MFA Rules not processed for user `c48b588bf82b4eb9ae2b6d7f0b137e3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:12.484054 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-b418eddd-5190-4e01-b3d1-dc991dc3139c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:12.501978 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2378cf3d-9f09-458b-bc92-2c7ef2b121e6 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:12.527740 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:12.573137 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 859/1701] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:12.574019 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b418eddd-5190-4e01-b3d1-dc991dc3139c None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:12.644278 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddcf303-df6d-429e-b6dc-649b5d72109a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:12.646147 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddcf303-df6d-429e-b6dc-649b5d72109a service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:12.646549 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddcf303-df6d-429e-b6dc-649b5d72109a service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:12.648091 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddcf303-df6d-429e-b6dc-649b5d72109a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:12.652572 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-eddcf303-df6d-429e-b6dc-649b5d72109a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:12.728564 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b418eddd-5190-4e01-b3d1-dc991dc3139c None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:12.744736 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df03b15-9073-4c69-8d72-632b4aa98fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:12.745104 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df03b15-9073-4c69-8d72-632b4aa98fda admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:12.745392 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df03b15-9073-4c69-8d72-632b4aa98fda admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:12.745725 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df03b15-9073-4c69-8d72-632b4aa98fda admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:12.746482 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df03b15-9073-4c69-8d72-632b4aa98fda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:12.748467 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df03b15-9073-4c69-8d72-632b4aa98fda admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:12.772131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddcf303-df6d-429e-b6dc-649b5d72109a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:12.774206 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddcf303-df6d-429e-b6dc-649b5d72109a service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:12.799741 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 860/1702] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:03:12 2022] POST /identity/v3/users => generated 289 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:12.799741 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df03b15-9073-4c69-8d72-632b4aa98fda admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:12.807653 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 843/1703] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:12 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:03:12.809221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddcf303-df6d-429e-b6dc-649b5d72109a service cinder] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:12.819883 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df03b15-9073-4c69-8d72-632b4aa98fda admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:12.844084 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e875040-9e77-4f8d-8890-f9b5b3e5a45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:12.844791 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e875040-9e77-4f8d-8890-f9b5b3e5a45f admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:12.845059 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e875040-9e77-4f8d-8890-f9b5b3e5a45f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:12.845359 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e875040-9e77-4f8d-8890-f9b5b3e5a45f admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:12.846054 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e875040-9e77-4f8d-8890-f9b5b3e5a45f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:12.847892 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddcf303-df6d-429e-b6dc-649b5d72109a service cinder] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:12.849744 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e875040-9e77-4f8d-8890-f9b5b3e5a45f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:12.867253 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ba2ae-4996-4c63-996f-be13730179a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:12.868039 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ba2ae-4996-4c63-996f-be13730179a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:12.868340 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ba2ae-4996-4c63-996f-be13730179a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:12.868638 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ba2ae-4996-4c63-996f-be13730179a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:12.869421 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1ba2ae-4996-4c63-996f-be13730179a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:12.886687 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1ba2ae-4996-4c63-996f-be13730179a0 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:12.915710 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 861/1704] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:03:12 2022] POST /identity/v3/projects => generated 381 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:12.921457 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e875040-9e77-4f8d-8890-f9b5b3e5a45f admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:12.926122 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 844/1705] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:12 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:12.931108 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ba2ae-4996-4c63-996f-be13730179a0 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:12.954225 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ba2ae-4996-4c63-996f-be13730179a0 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:13.170349 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e875040-9e77-4f8d-8890-f9b5b3e5a45f admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:13.185525 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae9016-f1ef-4a18-861b-c54555e82ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:13.186493 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae9016-f1ef-4a18-861b-c54555e82ea3 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:13.186788 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae9016-f1ef-4a18-861b-c54555e82ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:13.187066 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae9016-f1ef-4a18-861b-c54555e82ea3 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:13.187622 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d845a-41c8-4a8a-8206-0856703a17b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:13.188071 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ae9016-f1ef-4a18-861b-c54555e82ea3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:13.188834 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d845a-41c8-4a8a-8206-0856703a17b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:13.188834 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d845a-41c8-4a8a-8206-0856703a17b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:13.189091 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d845a-41c8-4a8a-8206-0856703a17b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:13.190458 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8d845a-41c8-4a8a-8206-0856703a17b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:13.195497 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8d845a-41c8-4a8a-8206-0856703a17b5 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:13.203465 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ae9016-f1ef-4a18-861b-c54555e82ea3 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:13.205752 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:13.238497 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 845/1706] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:12 2022] GET /identity/v3/roles => generated 1420 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:13.253897 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 862/1707] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:03:12 2022] POST /identity/v3/users => generated 297 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:13.259277 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae9016-f1ef-4a18-861b-c54555e82ea3 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:13.270580 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d845a-41c8-4a8a-8206-0856703a17b5 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:13.288289 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae9016-f1ef-4a18-861b-c54555e82ea3 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:13.302643 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d845a-41c8-4a8a-8206-0856703a17b5 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:13.303657 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eaa0a0-569a-4836-a321-1d85406797cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:13.308978 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eaa0a0-569a-4836-a321-1d85406797cb admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:13.309535 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eaa0a0-569a-4836-a321-1d85406797cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:13.310564 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eaa0a0-569a-4836-a321-1d85406797cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:13.312699 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eaa0a0-569a-4836-a321-1d85406797cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:13.316405 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eaa0a0-569a-4836-a321-1d85406797cb admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:13.329199 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaef0fff-d31a-409c-992e-8f5a3da04951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:13.331303 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef0fff-d31a-409c-992e-8f5a3da04951 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:13.334094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef0fff-d31a-409c-992e-8f5a3da04951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:13.334422 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef0fff-d31a-409c-992e-8f5a3da04951 admin admin] PATH_INFO: `/v3/projects/fec79ef616644f82ba670e3962dd0e4c/users/7bb7c70d094149fd9c3234e9c237faec/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:13.341773 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 863/1708] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:13 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:13.346472 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eaa0a0-569a-4836-a321-1d85406797cb admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:13.378333 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eaa0a0-569a-4836-a321-1d85406797cb admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:13.393346 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc5a1b8-5788-4dd5-84cc-f9f1859b7319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:13.394337 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc5a1b8-5788-4dd5-84cc-f9f1859b7319 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:13.394617 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc5a1b8-5788-4dd5-84cc-f9f1859b7319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:13.394897 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc5a1b8-5788-4dd5-84cc-f9f1859b7319 admin admin] PATH_INFO: `/v3/projects/17fc76ad16234a08a7e0a7dab5de59a9/users/7cdddfc252ad4457b5bd20deaed93c2c/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:13.395662 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaef0fff-d31a-409c-992e-8f5a3da04951 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fec79ef616644f82ba670e3962dd0e4c, user_id=7bb7c70d094149fd9c3234e9c237faec, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:13.399292 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaef0fff-d31a-409c-992e-8f5a3da04951 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:13.431468 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 846/1709] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:03:13 2022] PUT /identity/v3/projects/fec79ef616644f82ba670e3962dd0e4c/users/7bb7c70d094149fd9c3234e9c237faec/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:13.443668 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b3da7-73b7-4fd7-8fda-7546b068ae27 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:13.443958 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b3da7-73b7-4fd7-8fda-7546b068ae27 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:13.444830 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b3da7-73b7-4fd7-8fda-7546b068ae27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:13.446575 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc5a1b8-5788-4dd5-84cc-f9f1859b7319 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17fc76ad16234a08a7e0a7dab5de59a9, user_id=7cdddfc252ad4457b5bd20deaed93c2c, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:13.454071 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc5a1b8-5788-4dd5-84cc-f9f1859b7319 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:13.460202 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:13.484002 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 864/1710] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:03:13 2022] PUT /identity/v3/projects/17fc76ad16234a08a7e0a7dab5de59a9/users/7cdddfc252ad4457b5bd20deaed93c2c/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:13.485153 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc5a1b8-5788-4dd5-84cc-f9f1859b7319 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:13.525659 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-a48b3da7-73b7-4fd7-8fda-7546b068ae27 None None] MFA Rules not processed for user `7bb7c70d094149fd9c3234e9c237faec`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:13.527530 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-a48b3da7-73b7-4fd7-8fda-7546b068ae27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:13.616776 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc5a1b8-5788-4dd5-84cc-f9f1859b7319 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:13.627594 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a1f598-055d-41d4-ac72-7052908e2a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:13.628433 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1f598-055d-41d4-ac72-7052908e2a34 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:13.628722 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1f598-055d-41d4-ac72-7052908e2a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:13.629069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1f598-055d-41d4-ac72-7052908e2a34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:13.634018 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a1f598-055d-41d4-ac72-7052908e2a34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:13.635857 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a1f598-055d-41d4-ac72-7052908e2a34 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:13.658827 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 865/1711] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:13 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:13.660921 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a1f598-055d-41d4-ac72-7052908e2a34 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:13.704104 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a1f598-055d-41d4-ac72-7052908e2a34 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:13.716294 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecfaeac-c574-4009-accf-118e848f5f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:13.717241 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecfaeac-c574-4009-accf-118e848f5f26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:13.717448 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecfaeac-c574-4009-accf-118e848f5f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:13.717707 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecfaeac-c574-4009-accf-118e848f5f26 admin admin] PATH_INFO: `/v3/projects/17fc76ad16234a08a7e0a7dab5de59a9/users/7cdddfc252ad4457b5bd20deaed93c2c/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:13.722011 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecfaeac-c574-4009-accf-118e848f5f26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17fc76ad16234a08a7e0a7dab5de59a9, user_id=7cdddfc252ad4457b5bd20deaed93c2c, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:13.724914 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecfaeac-c574-4009-accf-118e848f5f26 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:13.729197 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 847/1712] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:13.748696 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48b3da7-73b7-4fd7-8fda-7546b068ae27 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:13.754261 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 866/1713] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:03:13 2022] PUT /identity/v3/projects/17fc76ad16234a08a7e0a7dab5de59a9/users/7cdddfc252ad4457b5bd20deaed93c2c/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:13.764794 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889b0db-8306-423f-ab92-8b0ee0fa6046 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:13.765087 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889b0db-8306-423f-ab92-8b0ee0fa6046 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:13.765372 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889b0db-8306-423f-ab92-8b0ee0fa6046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:13.772768 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48b3da7-73b7-4fd7-8fda-7546b068ae27 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:13.834282 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-8889b0db-8306-423f-ab92-8b0ee0fa6046 None None] MFA Rules not processed for user `7cdddfc252ad4457b5bd20deaed93c2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:13.835747 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-8889b0db-8306-423f-ab92-8b0ee0fa6046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:13.936092 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c286cade-3127-46f1-a812-7ffb18154496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:13.939028 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286cade-3127-46f1-a812-7ffb18154496 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:13.940487 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286cade-3127-46f1-a812-7ffb18154496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:13.941789 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286cade-3127-46f1-a812-7ffb18154496 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:13.944955 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c286cade-3127-46f1-a812-7ffb18154496 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:13.950264 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c286cade-3127-46f1-a812-7ffb18154496 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:14.018758 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 848/1715] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:03:13 2022] POST /identity/v3/projects => generated 375 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:14.024308 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb854a-6f70-4401-98e2-e2932c7ecb79 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:14.024835 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb854a-6f70-4401-98e2-e2932c7ecb79 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:14.025453 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb854a-6f70-4401-98e2-e2932c7ecb79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:14.026366 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 867/1715] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:13 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:14.028405 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8889b0db-8306-423f-ab92-8b0ee0fa6046 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:14.054213 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8889b0db-8306-423f-ab92-8b0ee0fa6046 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:14.109429 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-b3eb854a-6f70-4401-98e2-e2932c7ecb79 None None] MFA Rules not processed for user `c44c1788959e4b339c9cd5aad4318496`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:14.112914 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-b3eb854a-6f70-4401-98e2-e2932c7ecb79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:14.197121 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5abf21-72b4-4e98-9b78-e232a3778186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:14.197943 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5abf21-72b4-4e98-9b78-e232a3778186 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:14.198414 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5abf21-72b4-4e98-9b78-e232a3778186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:14.198880 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5abf21-72b4-4e98-9b78-e232a3778186 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:14.199718 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5abf21-72b4-4e98-9b78-e232a3778186 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:14.204170 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5abf21-72b4-4e98-9b78-e232a3778186 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:14.231831 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:14.259101 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 868/1716] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:03:14 2022] POST /identity/v3/users => generated 294 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:14.265124 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5abf21-72b4-4e98-9b78-e232a3778186 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:14.284500 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:14.384920 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 849/1717] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:14.387489 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eb854a-6f70-4401-98e2-e2932c7ecb79 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:14.391974 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-b3eb854a-6f70-4401-98e2-e2932c7ecb79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:14.447746 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eb854a-6f70-4401-98e2-e2932c7ecb79 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:14.461987 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb877ff3-758c-46cb-b3df-0af9458fc07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:14.464194 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb877ff3-758c-46cb-b3df-0af9458fc07b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:14.465677 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb877ff3-758c-46cb-b3df-0af9458fc07b admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:14.467140 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb877ff3-758c-46cb-b3df-0af9458fc07b admin admin] PATH_INFO: `/v3/projects/4637b625bf8245038c9606f3e4b0bcd4` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:14.470254 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb877ff3-758c-46cb-b3df-0af9458fc07b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4637b625bf8245038c9606f3e4b0bcd4)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:14.472970 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5abf21-72b4-4e98-9b78-e232a3778186 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:14.474453 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb877ff3-758c-46cb-b3df-0af9458fc07b admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:14.489779 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b82934-fa11-44d9-af46-861820a18099 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:14.490453 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b82934-fa11-44d9-af46-861820a18099 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:14.490734 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b82934-fa11-44d9-af46-861820a18099 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:14.491030 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b82934-fa11-44d9-af46-861820a18099 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:14.616525 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b82934-fa11-44d9-af46-861820a18099 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:14.617737 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b82934-fa11-44d9-af46-861820a18099 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:14.619792 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-cb877ff3-758c-46cb-b3df-0af9458fc07b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4637b625bf8245038c9606f3e4b0bcd4', 'request_id': 'req-cb877ff3-758c-46cb-b3df-0af9458fc07b'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:14.634560 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-cb877ff3-758c-46cb-b3df-0af9458fc07b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4637b625bf8245038c9606f3e4b0bcd4', 'request_id': 'req-cb877ff3-758c-46cb-b3df-0af9458fc07b'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:14.643835 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-cb877ff3-758c-46cb-b3df-0af9458fc07b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4637b625bf8245038c9606f3e4b0bcd4', 'request_id': 'req-cb877ff3-758c-46cb-b3df-0af9458fc07b'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:14.653104 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 869/1718] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:14 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:03:14.655298 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b82934-fa11-44d9-af46-861820a18099 service cinder] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:14.688863 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b82934-fa11-44d9-af46-861820a18099 service cinder] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:14.700292 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-cb877ff3-758c-46cb-b3df-0af9458fc07b admin admin] The token cache is being invalidate because project 4637b625bf8245038c9606f3e4b0bcd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:03:14.700644 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-cb877ff3-758c-46cb-b3df-0af9458fc07b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:14.706063 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-cb877ff3-758c-46cb-b3df-0af9458fc07b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:14.711480 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 850/1719] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:03:14 2022] DELETE /identity/v3/projects/4637b625bf8245038c9606f3e4b0bcd4 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:14.719746 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83861ef-8607-4772-a6db-b752beedad05 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:14.720045 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83861ef-8607-4772-a6db-b752beedad05 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:14.720329 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83861ef-8607-4772-a6db-b752beedad05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:14.785692 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-a83861ef-8607-4772-a6db-b752beedad05 None None] MFA Rules not processed for user `c44c1788959e4b339c9cd5aad4318496`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:14.787030 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-a83861ef-8607-4772-a6db-b752beedad05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:14.898357 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59430ca4-3e85-4c42-95fa-08090a252eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:14.899047 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59430ca4-3e85-4c42-95fa-08090a252eac admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:14.899412 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59430ca4-3e85-4c42-95fa-08090a252eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:14.899632 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59430ca4-3e85-4c42-95fa-08090a252eac admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:14.900746 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59430ca4-3e85-4c42-95fa-08090a252eac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:14.905689 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59430ca4-3e85-4c42-95fa-08090a252eac admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:14.935802 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 870/1720] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:14 2022] GET /identity/v3/roles => generated 1420 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:14.935802 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847e1e8b-1086-4a27-ab1b-34f687f4c79c None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:14.935802 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847e1e8b-1086-4a27-ab1b-34f687f4c79c None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:14.935802 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847e1e8b-1086-4a27-ab1b-34f687f4c79c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:14.994918 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 851/1721] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:14.997936 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83861ef-8607-4772-a6db-b752beedad05 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:14.998592 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-847e1e8b-1086-4a27-ab1b-34f687f4c79c None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:15.000102 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-847e1e8b-1086-4a27-ab1b-34f687f4c79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:15.000294 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-a83861ef-8607-4772-a6db-b752beedad05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:15.055031 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83861ef-8607-4772-a6db-b752beedad05 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:15.057389 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 871/1722] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:15.060166 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847e1e8b-1086-4a27-ab1b-34f687f4c79c None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:15.062384 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-847e1e8b-1086-4a27-ab1b-34f687f4c79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:15.082598 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1e4256-f0c3-45b2-a220-d1b0098d9a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:15.083568 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e4256-f0c3-45b2-a220-d1b0098d9a31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:15.083854 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e4256-f0c3-45b2-a220-d1b0098d9a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:15.084140 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e4256-f0c3-45b2-a220-d1b0098d9a31 admin admin] PATH_INFO: `/v3/projects/c393b6a3ec3141c3aff392278b481b31/users/695f4a75ceb74be0923ab8a1a30fb76b/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:15.146605 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1e4256-f0c3-45b2-a220-d1b0098d9a31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c393b6a3ec3141c3aff392278b481b31, user_id=695f4a75ceb74be0923ab8a1a30fb76b, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:15.159923 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1e4256-f0c3-45b2-a220-d1b0098d9a31 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:15.192841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:15.204891 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 852/1723] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:03:14 2022] PUT /identity/v3/projects/c393b6a3ec3141c3aff392278b481b31/users/695f4a75ceb74be0923ab8a1a30fb76b/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:15.205103 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51345339-1870-4390-ab34-dca244006fd9 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:15.205404 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51345339-1870-4390-ab34-dca244006fd9 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:15.205792 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51345339-1870-4390-ab34-dca244006fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:15.274605 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-51345339-1870-4390-ab34-dca244006fd9 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:15.275866 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-51345339-1870-4390-ab34-dca244006fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:15.291411 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847e1e8b-1086-4a27-ab1b-34f687f4c79c None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:15.398899 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:15.452947 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 853/1724] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:15.453677 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51345339-1870-4390-ab34-dca244006fd9 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:15.476406 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51345339-1870-4390-ab34-dca244006fd9 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:15.490400 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a8b78-4075-4c58-8af3-42217991a2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:15.491824 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a8b78-4075-4c58-8af3-42217991a2d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:15.491824 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a8b78-4075-4c58-8af3-42217991a2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:15.494136 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a8b78-4075-4c58-8af3-42217991a2d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:15.498014 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41a8b78-4075-4c58-8af3-42217991a2d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:15.500994 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41a8b78-4075-4c58-8af3-42217991a2d0 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:15.537726 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21601d1-cf26-48fb-b35d-004d50d84064 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:15.538324 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 854/1725] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:15 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:15.539047 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21601d1-cf26-48fb-b35d-004d50d84064 service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:15.540383 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21601d1-cf26-48fb-b35d-004d50d84064 service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:15.542028 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21601d1-cf26-48fb-b35d-004d50d84064 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:15.545244 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad8ab7-b576-4807-8be4-71cfae4eaa7a None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:15.545971 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad8ab7-b576-4807-8be4-71cfae4eaa7a None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:15.547856 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad8ab7-b576-4807-8be4-71cfae4eaa7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:15.550422 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-f21601d1-cf26-48fb-b35d-004d50d84064 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:15.616951 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-faad8ab7-b576-4807-8be4-71cfae4eaa7a None None] MFA Rules not processed for user `86619a80df8841a3ba7d8471ac38056a`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:15.619833 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-faad8ab7-b576-4807-8be4-71cfae4eaa7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:15.712220 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21601d1-cf26-48fb-b35d-004d50d84064 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:15.717647 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21601d1-cf26-48fb-b35d-004d50d84064 service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:15.739434 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 872/1726] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:15 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 681 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:03:15.741667 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21601d1-cf26-48fb-b35d-004d50d84064 service nova] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:15.756451 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 855/1727] 10.222.0.13 () {58 vars in 1009 bytes} [Tue Jun 7 22:03:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:15.757539 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faad8ab7-b576-4807-8be4-71cfae4eaa7a None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:15.772297 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21601d1-cf26-48fb-b35d-004d50d84064 service nova] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:15.779771 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faad8ab7-b576-4807-8be4-71cfae4eaa7a None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:15.784874 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034e8e3c-82e7-4fc9-a408-04e81a47bada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:15.785949 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e8e3c-82e7-4fc9-a408-04e81a47bada admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:15.787055 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e8e3c-82e7-4fc9-a408-04e81a47bada admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:15.787919 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e8e3c-82e7-4fc9-a408-04e81a47bada admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:15.789387 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034e8e3c-82e7-4fc9-a408-04e81a47bada admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:03:15.790290 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034e8e3c-82e7-4fc9-a408-04e81a47bada admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:15.792783 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034e8e3c-82e7-4fc9-a408-04e81a47bada admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:15.797540 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a563940-6078-4896-8d3e-f3aebb29f40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:15.798900 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a563940-6078-4896-8d3e-f3aebb29f40c admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:15.799296 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a563940-6078-4896-8d3e-f3aebb29f40c admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:15.799702 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a563940-6078-4896-8d3e-f3aebb29f40c admin admin] PATH_INFO: `/v3/projects/c393b6a3ec3141c3aff392278b481b31/users/695f4a75ceb74be0923ab8a1a30fb76b/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:15.806598 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a563940-6078-4896-8d3e-f3aebb29f40c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c393b6a3ec3141c3aff392278b481b31, user_id=695f4a75ceb74be0923ab8a1a30fb76b, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:15.810463 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a563940-6078-4896-8d3e-f3aebb29f40c admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:15.821179 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 873/1728] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:03:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:03:15.829360 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034e8e3c-82e7-4fc9-a408-04e81a47bada admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:15.833069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-034e8e3c-82e7-4fc9-a408-04e81a47bada admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:15.846673 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 856/1729] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:03:15 2022] PUT /identity/v3/projects/c393b6a3ec3141c3aff392278b481b31/users/695f4a75ceb74be0923ab8a1a30fb76b/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:15.847760 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a563940-6078-4896-8d3e-f3aebb29f40c admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:15.873664 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034e8e3c-82e7-4fc9-a408-04e81a47bada admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:15.999699 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:16.006197 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a563940-6078-4896-8d3e-f3aebb29f40c admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:16.016416 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c43f18a-9998-4ca2-8ab4-74438f9f0f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:16.017508 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c43f18a-9998-4ca2-8ab4-74438f9f0f2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:16.018582 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c43f18a-9998-4ca2-8ab4-74438f9f0f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:16.019612 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab66ba4-d386-41c5-a8c2-9e6f0a3c641f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:16.020061 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c43f18a-9998-4ca2-8ab4-74438f9f0f2b admin admin] PATH_INFO: `/v3/users/c48b588bf82b4eb9ae2b6d7f0b137e3f` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:16.021369 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab66ba4-d386-41c5-a8c2-9e6f0a3c641f admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:16.022212 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c43f18a-9998-4ca2-8ab4-74438f9f0f2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c48b588bf82b4eb9ae2b6d7f0b137e3f)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:16.022599 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab66ba4-d386-41c5-a8c2-9e6f0a3c641f admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:16.022981 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab66ba4-d386-41c5-a8c2-9e6f0a3c641f admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:16.024429 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab66ba4-d386-41c5-a8c2-9e6f0a3c641f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:16.026454 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c43f18a-9998-4ca2-8ab4-74438f9f0f2b admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:16.030348 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab66ba4-d386-41c5-a8c2-9e6f0a3c641f admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:16.090487 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 857/1730] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:03:15 2022] POST /identity/v3/projects => generated 369 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:16.097390 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c45b06-449d-4c5f-b0ce-b16d59ee269d None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:16.097939 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c45b06-449d-4c5f-b0ce-b16d59ee269d None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:16.098407 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c45b06-449d-4c5f-b0ce-b16d59ee269d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:16.135182 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-9c43f18a-9998-4ca2-8ab4-74438f9f0f2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c48b588bf82b4eb9ae2b6d7f0b137e3f'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:16.159428 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-9c43f18a-9998-4ca2-8ab4-74438f9f0f2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c48b588bf82b4eb9ae2b6d7f0b137e3f'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:16.174444 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-09c45b06-449d-4c5f-b0ce-b16d59ee269d None None] MFA Rules not processed for user `695f4a75ceb74be0923ab8a1a30fb76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:16.178316 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-09c45b06-449d-4c5f-b0ce-b16d59ee269d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:16.195985 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-9c43f18a-9998-4ca2-8ab4-74438f9f0f2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c48b588bf82b4eb9ae2b6d7f0b137e3f'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:16.199122 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-9c43f18a-9998-4ca2-8ab4-74438f9f0f2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c48b588bf82b4eb9ae2b6d7f0b137e3f'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:16.199833 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-9c43f18a-9998-4ca2-8ab4-74438f9f0f2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c48b588bf82b4eb9ae2b6d7f0b137e3f'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:16.234064 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 874/1731] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:03:15 2022] DELETE /identity/v3/users/c48b588bf82b4eb9ae2b6d7f0b137e3f => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:16.234064 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c43f18a-9998-4ca2-8ab4-74438f9f0f2b admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:16.237880 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-9c43f18a-9998-4ca2-8ab4-74438f9f0f2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:16.286431 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:16.366092 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 858/1732] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:16 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:16.366092 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c45b06-449d-4c5f-b0ce-b16d59ee269d None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:16.368277 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-09c45b06-449d-4c5f-b0ce-b16d59ee269d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:16.402977 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c43f18a-9998-4ca2-8ab4-74438f9f0f2b admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:16.418455 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f49f2ab-8159-455e-81ed-9d12176a7c53 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1261563b6d4e50a4ffeea6a32bdb01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:16.424387 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f49f2ab-8159-455e-81ed-9d12176a7c53 service swift] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:16.424766 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f49f2ab-8159-455e-81ed-9d12176a7c53 service swift] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:16.425197 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f49f2ab-8159-455e-81ed-9d12176a7c53 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:16.427856 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-1f49f2ab-8159-455e-81ed-9d12176a7c53 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:16.521412 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c45b06-449d-4c5f-b0ce-b16d59ee269d None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:16.571236 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f49f2ab-8159-455e-81ed-9d12176a7c53 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:16.572911 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f49f2ab-8159-455e-81ed-9d12176a7c53 service swift] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:16.598726 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 875/1733] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Jun 7 22:03:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 22:03:16.600455 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d82ac-a71e-44a9-a97a-92a7c64bf14f None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:16.600978 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d82ac-a71e-44a9-a97a-92a7c64bf14f None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:16.601257 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d82ac-a71e-44a9-a97a-92a7c64bf14f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:16.640679 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38a8c33-d3a8-4768-823b-108332519e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:16.643545 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a8c33-d3a8-4768-823b-108332519e2d admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:16.644267 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a8c33-d3a8-4768-823b-108332519e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:16.645422 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a8c33-d3a8-4768-823b-108332519e2d admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:16.646246 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38a8c33-d3a8-4768-823b-108332519e2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:16.652430 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38a8c33-d3a8-4768-823b-108332519e2d admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:16.653817 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-090d82ac-a71e-44a9-a97a-92a7c64bf14f None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:16.657642 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-090d82ac-a71e-44a9-a97a-92a7c64bf14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:16.677574 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 876/1734] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:16.678020 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d82ac-a71e-44a9-a97a-92a7c64bf14f None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:16.687386 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-090d82ac-a71e-44a9-a97a-92a7c64bf14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:16.709684 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 859/1735] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:03:16 2022] POST /identity/v3/users => generated 285 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:16.713365 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38a8c33-d3a8-4768-823b-108332519e2d admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:16.718192 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:16.735781 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38a8c33-d3a8-4768-823b-108332519e2d admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:16.752357 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66188bc-f642-4393-94f8-25ce2a068dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:16.753142 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66188bc-f642-4393-94f8-25ce2a068dab admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:16.757879 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66188bc-f642-4393-94f8-25ce2a068dab admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:16.757879 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66188bc-f642-4393-94f8-25ce2a068dab admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:16.757879 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66188bc-f642-4393-94f8-25ce2a068dab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:16.757879 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66188bc-f642-4393-94f8-25ce2a068dab admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:16.810668 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 860/1736] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:16 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:16.811713 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66188bc-f642-4393-94f8-25ce2a068dab admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:16.875868 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66188bc-f642-4393-94f8-25ce2a068dab admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:16.882593 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d82ac-a71e-44a9-a97a-92a7c64bf14f None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:16.894522 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3e0a24-5222-475b-a893-8250c0ccb2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:16.902503 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e0a24-5222-475b-a893-8250c0ccb2a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:16.903559 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8ca92f-6b09-4bbb-9f85-7a2d628c12c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:16.904226 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8ca92f-6b09-4bbb-9f85-7a2d628c12c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:16.904765 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8ca92f-6b09-4bbb-9f85-7a2d628c12c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:16.905056 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8ca92f-6b09-4bbb-9f85-7a2d628c12c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:16.912500 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e0a24-5222-475b-a893-8250c0ccb2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:16.912797 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e0a24-5222-475b-a893-8250c0ccb2a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:16.913982 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-6a8ca92f-6b09-4bbb-9f85-7a2d628c12c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:16.922744 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3e0a24-5222-475b-a893-8250c0ccb2a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:16.942238 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3e0a24-5222-475b-a893-8250c0ccb2a7 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:17.004186 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 861/1737] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:16 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:17.005248 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3e0a24-5222-475b-a893-8250c0ccb2a7 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:17.012632 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-fb3e0a24-5222-475b-a893-8250c0ccb2a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:17.067962 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3e0a24-5222-475b-a893-8250c0ccb2a7 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:17.087347 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8311a068-f9d6-4882-a612-f365a24657f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:17.088431 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311a068-f9d6-4882-a612-f365a24657f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:17.088752 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311a068-f9d6-4882-a612-f365a24657f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:17.089078 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311a068-f9d6-4882-a612-f365a24657f7 admin admin] PATH_INFO: `/v3/users/93604b93b7df47aba313bebca0c99fb1` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:17.092769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8311a068-f9d6-4882-a612-f365a24657f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93604b93b7df47aba313bebca0c99fb1)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:17.096257 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8311a068-f9d6-4882-a612-f365a24657f7 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:17.176737 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:17.182949 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-8311a068-f9d6-4882-a612-f365a24657f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93604b93b7df47aba313bebca0c99fb1'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:17.184152 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-8311a068-f9d6-4882-a612-f365a24657f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93604b93b7df47aba313bebca0c99fb1'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:17.200433 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-8311a068-f9d6-4882-a612-f365a24657f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93604b93b7df47aba313bebca0c99fb1'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:17.210565 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8ca92f-6b09-4bbb-9f85-7a2d628c12c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:17.220546 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8ca92f-6b09-4bbb-9f85-7a2d628c12c1 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:17.222288 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-8311a068-f9d6-4882-a612-f365a24657f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93604b93b7df47aba313bebca0c99fb1'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:17.238916 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-8311a068-f9d6-4882-a612-f365a24657f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93604b93b7df47aba313bebca0c99fb1'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:17.252968 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 862/1738] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:03:17 2022] DELETE /identity/v3/users/93604b93b7df47aba313bebca0c99fb1 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:17.254364 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8311a068-f9d6-4882-a612-f365a24657f7 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:17.260861 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 877/1739] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:16 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 583 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:03:17.261548 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-8311a068-f9d6-4882-a612-f365a24657f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:17.264242 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8ca92f-6b09-4bbb-9f85-7a2d628c12c1 service cinder] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:17.309367 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8ca92f-6b09-4bbb-9f85-7a2d628c12c1 service cinder] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:17.456731 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8311a068-f9d6-4882-a612-f365a24657f7 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:17.470376 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209e654c-7aa1-425c-917c-79d53ef4a566 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:17.471082 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e654c-7aa1-425c-917c-79d53ef4a566 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:17.471389 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e654c-7aa1-425c-917c-79d53ef4a566 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:17.471676 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e654c-7aa1-425c-917c-79d53ef4a566 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:17.479263 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-209e654c-7aa1-425c-917c-79d53ef4a566 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:17.525088 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c99ec0-df80-4618-833a-38da8ee5f6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:17.532442 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c99ec0-df80-4618-833a-38da8ee5f6c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:17.534159 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c99ec0-df80-4618-833a-38da8ee5f6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:17.534798 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c99ec0-df80-4618-833a-38da8ee5f6c1 admin admin] PATH_INFO: `/v3/projects/10eea15a0d374e6fa9c1cab810111128/users/d441e66bc47a4446a893fffba3a9457e/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:17.585302 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c99ec0-df80-4618-833a-38da8ee5f6c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10eea15a0d374e6fa9c1cab810111128, user_id=d441e66bc47a4446a893fffba3a9457e, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:17.588515 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c99ec0-df80-4618-833a-38da8ee5f6c1 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:17.621164 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 878/1740] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:03:17 2022] PUT /identity/v3/projects/10eea15a0d374e6fa9c1cab810111128/users/d441e66bc47a4446a893fffba3a9457e/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:17.625630 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d6da2-deab-4fb8-acf2-def5ac1a7498 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:17.626010 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d6da2-deab-4fb8-acf2-def5ac1a7498 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:17.626393 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d6da2-deab-4fb8-acf2-def5ac1a7498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:17.680476 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-f40d6da2-deab-4fb8-acf2-def5ac1a7498 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:17.686034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-f40d6da2-deab-4fb8-acf2-def5ac1a7498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:17.747941 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209e654c-7aa1-425c-917c-79d53ef4a566 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:17.749409 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209e654c-7aa1-425c-917c-79d53ef4a566 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:17.806453 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:17.825427 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 863/1741] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:03:17.826955 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af72b6e-a35b-4658-9062-186db65ebc42 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:17.827252 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af72b6e-a35b-4658-9062-186db65ebc42 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:17.827572 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af72b6e-a35b-4658-9062-186db65ebc42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:17.863875 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 879/1742] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:17.872703 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3483975-3e36-4ab1-9698-2370106aadfd None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:17.872972 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3483975-3e36-4ab1-9698-2370106aadfd None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:17.873378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3483975-3e36-4ab1-9698-2370106aadfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:17.927154 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-c3483975-3e36-4ab1-9698-2370106aadfd None None] MFA Rules not processed for user `10242251e6284217bfb79c67b1567f75`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:17.933664 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c3483975-3e36-4ab1-9698-2370106aadfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:17.955481 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-3af72b6e-a35b-4658-9062-186db65ebc42 None None] MFA Rules not processed for user `d441e66bc47a4446a893fffba3a9457e`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:17.956948 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-3af72b6e-a35b-4658-9062-186db65ebc42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:18.137307 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 880/1743] 10.222.0.32 () {58 vars in 1015 bytes} [Tue Jun 7 22:03:17 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:18.142134 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3483975-3e36-4ab1-9698-2370106aadfd None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:18.193492 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3483975-3e36-4ab1-9698-2370106aadfd None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:18.200121 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:18.206701 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f53df-3473-4ef8-8614-25c864f9cd53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:18.207853 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f53df-3473-4ef8-8614-25c864f9cd53 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:18.210633 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f53df-3473-4ef8-8614-25c864f9cd53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:18.211096 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f53df-3473-4ef8-8614-25c864f9cd53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:18.220493 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 864/1744] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:17 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 394 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:18.225936 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af72b6e-a35b-4658-9062-186db65ebc42 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:18.248947 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91f53df-3473-4ef8-8614-25c864f9cd53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:18.254752 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91f53df-3473-4ef8-8614-25c864f9cd53 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:18.268275 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af72b6e-a35b-4658-9062-186db65ebc42 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:18.287973 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c972ddff-b2ac-4f6b-83bb-6d905540d01e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:18.289624 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972ddff-b2ac-4f6b-83bb-6d905540d01e service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:18.289920 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972ddff-b2ac-4f6b-83bb-6d905540d01e service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:18.291425 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972ddff-b2ac-4f6b-83bb-6d905540d01e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:18.297134 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 881/1745] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:03:18.304829 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f53df-3473-4ef8-8614-25c864f9cd53 service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:18.317192 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-b91f53df-3473-4ef8-8614-25c864f9cd53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:18.358469 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c972ddff-b2ac-4f6b-83bb-6d905540d01e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:18.360005 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c972ddff-b2ac-4f6b-83bb-6d905540d01e service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:18.421913 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 865/1746] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:03:18.433880 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f53df-3473-4ef8-8614-25c864f9cd53 service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:18.448632 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3ba89-9fb5-4009-a2b8-0a2e396842e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:18.450959 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3ba89-9fb5-4009-a2b8-0a2e396842e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:18.451303 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3ba89-9fb5-4009-a2b8-0a2e396842e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:18.452296 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3ba89-9fb5-4009-a2b8-0a2e396842e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:18.455893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf3ba89-9fb5-4009-a2b8-0a2e396842e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:18.459675 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf3ba89-9fb5-4009-a2b8-0a2e396842e1 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:18.574552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 882/1747] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:03:18 2022] POST /identity/v3/projects => generated 369 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:18.575720 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c972ddff-b2ac-4f6b-83bb-6d905540d01e service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:18.605727 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c972ddff-b2ac-4f6b-83bb-6d905540d01e service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:18.774872 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25125ece-ed80-4da9-bbd6-f54970aa0690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:18.776565 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25125ece-ed80-4da9-bbd6-f54970aa0690 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:18.777049 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25125ece-ed80-4da9-bbd6-f54970aa0690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:18.777511 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25125ece-ed80-4da9-bbd6-f54970aa0690 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:18.780140 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25125ece-ed80-4da9-bbd6-f54970aa0690 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:18.783526 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25125ece-ed80-4da9-bbd6-f54970aa0690 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:18.832710 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 866/1748] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:03:18 2022] POST /identity/v3/users => generated 291 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:18.837724 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3ba89-9fb5-4009-a2b8-0a2e396842e1 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:18.862701 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3ba89-9fb5-4009-a2b8-0a2e396842e1 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:18.868193 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9282c7-3e7a-4910-970e-91a2e84b56c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:18.868934 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9282c7-3e7a-4910-970e-91a2e84b56c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:18.869225 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9282c7-3e7a-4910-970e-91a2e84b56c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:18.869505 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9282c7-3e7a-4910-970e-91a2e84b56c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:18.884680 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9282c7-3e7a-4910-970e-91a2e84b56c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:18.895418 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9282c7-3e7a-4910-970e-91a2e84b56c4 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:18.914009 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 883/1749] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:18 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:18.914865 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25125ece-ed80-4da9-bbd6-f54970aa0690 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:18.930776 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25125ece-ed80-4da9-bbd6-f54970aa0690 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:18.957278 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe65e841-1cae-4bc3-9c63-df13381620ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:18.958344 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe65e841-1cae-4bc3-9c63-df13381620ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:18.958631 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe65e841-1cae-4bc3-9c63-df13381620ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:18.959052 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe65e841-1cae-4bc3-9c63-df13381620ec admin admin] PATH_INFO: `/v3/projects/3ee07fa4e1444a088b1a6650c9c233ca/users/fc36120c63934f6b96dc3aa3a926d3fd/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:19.018688 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe65e841-1cae-4bc3-9c63-df13381620ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ee07fa4e1444a088b1a6650c9c233ca, user_id=fc36120c63934f6b96dc3aa3a926d3fd, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:19.021748 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe65e841-1cae-4bc3-9c63-df13381620ec admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:19.103171 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 867/1750] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:03:18 2022] PUT /identity/v3/projects/3ee07fa4e1444a088b1a6650c9c233ca/users/fc36120c63934f6b96dc3aa3a926d3fd/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:19.104558 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9282c7-3e7a-4910-970e-91a2e84b56c4 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:19.122370 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9282c7-3e7a-4910-970e-91a2e84b56c4 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:19.325002 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d352fc9d-3191-4275-81b6-dffa52784e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:19.325849 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d352fc9d-3191-4275-81b6-dffa52784e3d admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:19.326360 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d352fc9d-3191-4275-81b6-dffa52784e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:19.327154 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d352fc9d-3191-4275-81b6-dffa52784e3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:19.327734 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d352fc9d-3191-4275-81b6-dffa52784e3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:19.329580 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d352fc9d-3191-4275-81b6-dffa52784e3d admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:19.368412 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe65e841-1cae-4bc3-9c63-df13381620ec admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:19.378353 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 884/1751] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:19 2022] GET /identity/v3/roles => generated 1420 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:19.385572 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe65e841-1cae-4bc3-9c63-df13381620ec admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:19.400323 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49581aca-5626-470f-b86b-f451d8e2070d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:19.401383 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49581aca-5626-470f-b86b-f451d8e2070d admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:19.402334 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49581aca-5626-470f-b86b-f451d8e2070d admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:19.402743 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49581aca-5626-470f-b86b-f451d8e2070d admin admin] PATH_INFO: `/v3/projects/3ee07fa4e1444a088b1a6650c9c233ca/users/fc36120c63934f6b96dc3aa3a926d3fd/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:19.409160 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49581aca-5626-470f-b86b-f451d8e2070d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ee07fa4e1444a088b1a6650c9c233ca, user_id=fc36120c63934f6b96dc3aa3a926d3fd, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:19.413665 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49581aca-5626-470f-b86b-f451d8e2070d admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:19.475394 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 868/1752] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:03:19 2022] PUT /identity/v3/projects/3ee07fa4e1444a088b1a6650c9c233ca/users/fc36120c63934f6b96dc3aa3a926d3fd/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:19.483860 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a0270b-b8ff-47d0-ba11-004be6fa0b24 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:19.484157 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a0270b-b8ff-47d0-ba11-004be6fa0b24 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:19.485185 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a0270b-b8ff-47d0-ba11-004be6fa0b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:19.648868 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-46a0270b-b8ff-47d0-ba11-004be6fa0b24 None None] MFA Rules not processed for user `fc36120c63934f6b96dc3aa3a926d3fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:19.653851 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-46a0270b-b8ff-47d0-ba11-004be6fa0b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:19.706298 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49581aca-5626-470f-b86b-f451d8e2070d admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:19.708316 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-49581aca-5626-470f-b86b-f451d8e2070d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:19.862748 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:19.936374 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49581aca-5626-470f-b86b-f451d8e2070d admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:19.953647 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178bba35-76c3-4038-9662-cbaf377bb1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:19.955036 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178bba35-76c3-4038-9662-cbaf377bb1de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:19.955652 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178bba35-76c3-4038-9662-cbaf377bb1de admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:19.956256 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178bba35-76c3-4038-9662-cbaf377bb1de admin admin] PATH_INFO: `/v3/projects/a2d5c5b32045404486f39af9c2598716` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:19.961106 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178bba35-76c3-4038-9662-cbaf377bb1de admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2d5c5b32045404486f39af9c2598716)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:19.967720 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178bba35-76c3-4038-9662-cbaf377bb1de admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:20.104910 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 885/1753] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:19 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 622 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:20.115493 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec47603-e157-4697-8f0d-d31bc7eb734c None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:20.115812 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec47603-e157-4697-8f0d-d31bc7eb734c None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:20.116100 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec47603-e157-4697-8f0d-d31bc7eb734c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:20.190667 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:20.210359 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-178bba35-76c3-4038-9662-cbaf377bb1de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2d5c5b32045404486f39af9c2598716', 'request_id': 'req-178bba35-76c3-4038-9662-cbaf377bb1de'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:20.212061 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-6ec47603-e157-4697-8f0d-d31bc7eb734c None None] MFA Rules not processed for user `c44c1788959e4b339c9cd5aad4318496`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:20.213404 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-6ec47603-e157-4697-8f0d-d31bc7eb734c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:20.231190 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-178bba35-76c3-4038-9662-cbaf377bb1de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2d5c5b32045404486f39af9c2598716', 'request_id': 'req-178bba35-76c3-4038-9662-cbaf377bb1de'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:20.251650 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-178bba35-76c3-4038-9662-cbaf377bb1de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2d5c5b32045404486f39af9c2598716', 'request_id': 'req-178bba35-76c3-4038-9662-cbaf377bb1de'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:20.280755 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-178bba35-76c3-4038-9662-cbaf377bb1de admin admin] The token cache is being invalidate because project a2d5c5b32045404486f39af9c2598716 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:03:20.281044 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-178bba35-76c3-4038-9662-cbaf377bb1de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:20.285511 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-178bba35-76c3-4038-9662-cbaf377bb1de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:20.292977 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 869/1754] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:03:19 2022] DELETE /identity/v3/projects/a2d5c5b32045404486f39af9c2598716 => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:20.292977 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178bba35-76c3-4038-9662-cbaf377bb1de admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:20.295549 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-178bba35-76c3-4038-9662-cbaf377bb1de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:20.499802 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 886/1755] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:20.503274 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec47603-e157-4697-8f0d-d31bc7eb734c None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:20.507913 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-6ec47603-e157-4697-8f0d-d31bc7eb734c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:20.510085 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178bba35-76c3-4038-9662-cbaf377bb1de admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:20.532285 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc1ef4-ec66-4b86-8787-c3b1a2bf16b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:20.537270 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc1ef4-ec66-4b86-8787-c3b1a2bf16b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:20.537721 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc1ef4-ec66-4b86-8787-c3b1a2bf16b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:20.538450 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc1ef4-ec66-4b86-8787-c3b1a2bf16b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:20.544898 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-6dcc1ef4-ec66-4b86-8787-c3b1a2bf16b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:20.800053 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcc1ef4-ec66-4b86-8787-c3b1a2bf16b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:20.802729 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcc1ef4-ec66-4b86-8787-c3b1a2bf16b8 service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:20.819172 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec47603-e157-4697-8f0d-d31bc7eb734c None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:20.836243 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f5a4ad-f6d6-49ee-b692-a298d147d9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:20.837143 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f5a4ad-f6d6-49ee-b692-a298d147d9fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:20.837419 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f5a4ad-f6d6-49ee-b692-a298d147d9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:20.837691 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f5a4ad-f6d6-49ee-b692-a298d147d9fd admin admin] PATH_INFO: `/v3/users/7cdddfc252ad4457b5bd20deaed93c2c` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:20.841611 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 870/1756] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:20 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:03:20.841765 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f5a4ad-f6d6-49ee-b692-a298d147d9fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cdddfc252ad4457b5bd20deaed93c2c)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:20.844296 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f5a4ad-f6d6-49ee-b692-a298d147d9fd admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:21.086941 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-f3f5a4ad-f6d6-49ee-b692-a298d147d9fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cdddfc252ad4457b5bd20deaed93c2c'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:21.118216 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-f3f5a4ad-f6d6-49ee-b692-a298d147d9fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cdddfc252ad4457b5bd20deaed93c2c'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:21.197534 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-f3f5a4ad-f6d6-49ee-b692-a298d147d9fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cdddfc252ad4457b5bd20deaed93c2c'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:21.204827 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-f3f5a4ad-f6d6-49ee-b692-a298d147d9fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cdddfc252ad4457b5bd20deaed93c2c'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:21.207349 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-f3f5a4ad-f6d6-49ee-b692-a298d147d9fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cdddfc252ad4457b5bd20deaed93c2c'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:21.231136 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:21.261788 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 887/1757] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:03:20 2022] DELETE /identity/v3/users/7cdddfc252ad4457b5bd20deaed93c2c => generated 0 bytes in 761 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:22.764539 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc1ef4-ec66-4b86-8787-c3b1a2bf16b8 service cinder] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:22.766500 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-6dcc1ef4-ec66-4b86-8787-c3b1a2bf16b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:22.795394 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc1ef4-ec66-4b86-8787-c3b1a2bf16b8 service cinder] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:22.820050 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ede4d-dc0c-4bf1-8cfb-0c89686c369f None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:22.820637 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ede4d-dc0c-4bf1-8cfb-0c89686c369f None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:22.820637 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ede4d-dc0c-4bf1-8cfb-0c89686c369f None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:22.821961 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 888/1758] 10.222.0.13 () {54 vars in 921 bytes} [Tue Jun 7 22:03:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 22:03:22.829439 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9ed4e-3e92-4807-99a4-02f585578647 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:22.829875 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9ed4e-3e92-4807-99a4-02f585578647 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:22.830306 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9ed4e-3e92-4807-99a4-02f585578647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:22.864250 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-23c9ed4e-3e92-4807-99a4-02f585578647 None None] MFA Rules not processed for user `dbe0303ea0764347bfb054870ebba41f`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:22.865493 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-23c9ed4e-3e92-4807-99a4-02f585578647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:22.964095 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9968391f-201e-4519-a170-56c6224f9561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:22.965049 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9968391f-201e-4519-a170-56c6224f9561 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:22.965456 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9968391f-201e-4519-a170-56c6224f9561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:22.965908 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9968391f-201e-4519-a170-56c6224f9561 admin admin] PATH_INFO: `/v3/projects/4b12be8645a94204936370294de6c1cb` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:22.968253 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9968391f-201e-4519-a170-56c6224f9561 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b12be8645a94204936370294de6c1cb)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:22.970169 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9968391f-201e-4519-a170-56c6224f9561 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:23.025129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:23.035672 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 889/1759] 10.222.0.13 () {58 vars in 1018 bytes} [Tue Jun 7 22:03:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:03:23.041491 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c9ed4e-3e92-4807-99a4-02f585578647 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:23.045128 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-23c9ed4e-3e92-4807-99a4-02f585578647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:23.081078 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c9ed4e-3e92-4807-99a4-02f585578647 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:23.085261 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-9968391f-201e-4519-a170-56c6224f9561 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b12be8645a94204936370294de6c1cb', 'request_id': 'req-9968391f-201e-4519-a170-56c6224f9561'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:23.104310 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-9968391f-201e-4519-a170-56c6224f9561 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b12be8645a94204936370294de6c1cb', 'request_id': 'req-9968391f-201e-4519-a170-56c6224f9561'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:23.123107 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-9968391f-201e-4519-a170-56c6224f9561 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b12be8645a94204936370294de6c1cb', 'request_id': 'req-9968391f-201e-4519-a170-56c6224f9561'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:23.153881 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-9968391f-201e-4519-a170-56c6224f9561 admin admin] The token cache is being invalidate because project 4b12be8645a94204936370294de6c1cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:03:23.154579 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-9968391f-201e-4519-a170-56c6224f9561 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:23.155644 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-9968391f-201e-4519-a170-56c6224f9561 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:23.157706 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 871/1760] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:03:22 2022] DELETE /identity/v3/projects/4b12be8645a94204936370294de6c1cb => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:23.159066 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9968391f-201e-4519-a170-56c6224f9561 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:23.161670 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-9968391f-201e-4519-a170-56c6224f9561 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:23.229373 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd81cbce-4387-4851-89be-7d4806fcd00a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:23.231354 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81cbce-4387-4851-89be-7d4806fcd00a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:23.231825 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81cbce-4387-4851-89be-7d4806fcd00a admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:23.232253 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81cbce-4387-4851-89be-7d4806fcd00a admin admin] PATH_INFO: `/v3/projects/17fc76ad16234a08a7e0a7dab5de59a9` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:23.252910 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd81cbce-4387-4851-89be-7d4806fcd00a admin admin] RBAC: Authorizing `identity:delete_project(project_id=17fc76ad16234a08a7e0a7dab5de59a9)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:23.254824 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd81cbce-4387-4851-89be-7d4806fcd00a admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:23.348967 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9968391f-201e-4519-a170-56c6224f9561 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:23.375803 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-bd81cbce-4387-4851-89be-7d4806fcd00a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17fc76ad16234a08a7e0a7dab5de59a9', 'request_id': 'req-bd81cbce-4387-4851-89be-7d4806fcd00a'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:23.394922 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-bd81cbce-4387-4851-89be-7d4806fcd00a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17fc76ad16234a08a7e0a7dab5de59a9', 'request_id': 'req-bd81cbce-4387-4851-89be-7d4806fcd00a'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:23.403879 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-bd81cbce-4387-4851-89be-7d4806fcd00a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17fc76ad16234a08a7e0a7dab5de59a9', 'request_id': 'req-bd81cbce-4387-4851-89be-7d4806fcd00a'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:23.424783 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-bd81cbce-4387-4851-89be-7d4806fcd00a admin admin] The token cache is being invalidate because project 17fc76ad16234a08a7e0a7dab5de59a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:03:23.425846 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-bd81cbce-4387-4851-89be-7d4806fcd00a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:23.427593 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-bd81cbce-4387-4851-89be-7d4806fcd00a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:23.436001 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 890/1761] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:03:23 2022] DELETE /identity/v3/projects/17fc76ad16234a08a7e0a7dab5de59a9 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:23.436001 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd81cbce-4387-4851-89be-7d4806fcd00a admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:23.499510 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-bd81cbce-4387-4851-89be-7d4806fcd00a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:23.515512 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1d86c5-54ab-4738-9926-fa2e3be87357 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:23.516284 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d86c5-54ab-4738-9926-fa2e3be87357 service nova] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:23.516590 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d86c5-54ab-4738-9926-fa2e3be87357 service nova] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:23.516993 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d86c5-54ab-4738-9926-fa2e3be87357 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:23.521313 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-3c1d86c5-54ab-4738-9926-fa2e3be87357 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:23.556279 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1d86c5-54ab-4738-9926-fa2e3be87357 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:23.558496 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1d86c5-54ab-4738-9926-fa2e3be87357 service nova] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:23.569902 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:23.588815 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 872/1762] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:03:23.605698 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901d2f7-6f15-40f9-9335-8710a8bdb203 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:23.606051 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901d2f7-6f15-40f9-9335-8710a8bdb203 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:23.606332 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901d2f7-6f15-40f9-9335-8710a8bdb203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:23.700194 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-1901d2f7-6f15-40f9-9335-8710a8bdb203 None None] MFA Rules not processed for user `607b70b4d6e14c95ae2880dd3577c8ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:23.702148 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-1901d2f7-6f15-40f9-9335-8710a8bdb203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:23.716029 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd81cbce-4387-4851-89be-7d4806fcd00a admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:23.726481 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2943a81f-4c4b-43df-86fe-7a9def8df156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:23.727759 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2943a81f-4c4b-43df-86fe-7a9def8df156 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:23.728536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2943a81f-4c4b-43df-86fe-7a9def8df156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:23.728994 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2943a81f-4c4b-43df-86fe-7a9def8df156 admin admin] PATH_INFO: `/v3/users/65718ab4d9be40fa83ef31498bc015fa` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:23.732012 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2943a81f-4c4b-43df-86fe-7a9def8df156 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65718ab4d9be40fa83ef31498bc015fa)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:23.735864 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2943a81f-4c4b-43df-86fe-7a9def8df156 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:23.773596 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:23.932063 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-2943a81f-4c4b-43df-86fe-7a9def8df156 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65718ab4d9be40fa83ef31498bc015fa'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:23.954558 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 873/1763] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:23.956280 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b631c-a4b6-4106-916e-affaa6aa4fe0 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:23.956582 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b631c-a4b6-4106-916e-affaa6aa4fe0 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:23.956852 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b631c-a4b6-4106-916e-affaa6aa4fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:23.974790 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-2943a81f-4c4b-43df-86fe-7a9def8df156 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65718ab4d9be40fa83ef31498bc015fa'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:23.993661 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-2943a81f-4c4b-43df-86fe-7a9def8df156 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65718ab4d9be40fa83ef31498bc015fa'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:23.994649 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-2943a81f-4c4b-43df-86fe-7a9def8df156 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65718ab4d9be40fa83ef31498bc015fa'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:23.996452 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-2943a81f-4c4b-43df-86fe-7a9def8df156 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65718ab4d9be40fa83ef31498bc015fa'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:24.017124 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-239b631c-a4b6-4106-916e-affaa6aa4fe0 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:24.018696 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-239b631c-a4b6-4106-916e-affaa6aa4fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:24.066534 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 874/1764] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:24.072740 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b631c-a4b6-4106-916e-affaa6aa4fe0 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:24.076185 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-239b631c-a4b6-4106-916e-affaa6aa4fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:24.085010 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 891/1765] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:03:23 2022] DELETE /identity/v3/users/65718ab4d9be40fa83ef31498bc015fa => generated 0 bytes in 654 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:24.096855 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab26cfa-c887-4879-bbd3-83a8f4cd2ac6 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:24.097194 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab26cfa-c887-4879-bbd3-83a8f4cd2ac6 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:24.097615 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab26cfa-c887-4879-bbd3-83a8f4cd2ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:24.131270 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b631c-a4b6-4106-916e-affaa6aa4fe0 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:24.158678 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-aab26cfa-c887-4879-bbd3-83a8f4cd2ac6 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:24.165507 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-aab26cfa-c887-4879-bbd3-83a8f4cd2ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:24.344565 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc830800-4aa9-4c93-9e63-bcdb7cb1061f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:24.347956 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc830800-4aa9-4c93-9e63-bcdb7cb1061f service nova] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:24.350235 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc830800-4aa9-4c93-9e63-bcdb7cb1061f service nova] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:24.351183 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc830800-4aa9-4c93-9e63-bcdb7cb1061f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:24.371761 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-fc830800-4aa9-4c93-9e63-bcdb7cb1061f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:24.437693 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 892/1766] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:24.440026 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab26cfa-c887-4879-bbd3-83a8f4cd2ac6 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:24.447454 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-aab26cfa-c887-4879-bbd3-83a8f4cd2ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:24.472461 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab26cfa-c887-4879-bbd3-83a8f4cd2ac6 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:24.498736 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b914e2-210e-4319-87e8-6a548be8286f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:24.499753 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b914e2-210e-4319-87e8-6a548be8286f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:24.500040 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b914e2-210e-4319-87e8-6a548be8286f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:24.500322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b914e2-210e-4319-87e8-6a548be8286f admin admin] PATH_INFO: `/v3/users/d441e66bc47a4446a893fffba3a9457e` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:24.507026 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b914e2-210e-4319-87e8-6a548be8286f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d441e66bc47a4446a893fffba3a9457e)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:24.509711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b914e2-210e-4319-87e8-6a548be8286f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:24.592542 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc830800-4aa9-4c93-9e63-bcdb7cb1061f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:24.593647 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc830800-4aa9-4c93-9e63-bcdb7cb1061f service nova] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:24.607423 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:24.630993 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 875/1767] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 559 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:03:24.632210 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc830800-4aa9-4c93-9e63-bcdb7cb1061f service nova] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:24.632637 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-09b914e2-210e-4319-87e8-6a548be8286f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd441e66bc47a4446a893fffba3a9457e'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:24.666743 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-09b914e2-210e-4319-87e8-6a548be8286f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd441e66bc47a4446a893fffba3a9457e'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:24.686022 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc830800-4aa9-4c93-9e63-bcdb7cb1061f service nova] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:24.701480 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-09b914e2-210e-4319-87e8-6a548be8286f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd441e66bc47a4446a893fffba3a9457e'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:24.716686 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-09b914e2-210e-4319-87e8-6a548be8286f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd441e66bc47a4446a893fffba3a9457e'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:24.716850 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ae904-7adc-4ce5-abb5-a623a774fe4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:24.717180 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ae904-7adc-4ce5-abb5-a623a774fe4b admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:24.717462 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ae904-7adc-4ce5-abb5-a623a774fe4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:24.717569 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-09b914e2-210e-4319-87e8-6a548be8286f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd441e66bc47a4446a893fffba3a9457e'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:24.717740 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ae904-7adc-4ce5-abb5-a623a774fe4b admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:24.718474 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765ae904-7adc-4ce5-abb5-a623a774fe4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:03:24.718790 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765ae904-7adc-4ce5-abb5-a623a774fe4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:24.721700 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765ae904-7adc-4ce5-abb5-a623a774fe4b admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:24.752916 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 893/1768] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:03:24 2022] DELETE /identity/v3/users/d441e66bc47a4446a893fffba3a9457e => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:24.768258 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 876/1769] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:03:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:03:24.770898 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877fc9ac-f068-4457-a5d4-90fb32e4a6be None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:24.771840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877fc9ac-f068-4457-a5d4-90fb32e4a6be None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:24.772393 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877fc9ac-f068-4457-a5d4-90fb32e4a6be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:24.775426 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ae904-7adc-4ce5-abb5-a623a774fe4b admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:24.782510 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-765ae904-7adc-4ce5-abb5-a623a774fe4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:24.840219 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-877fc9ac-f068-4457-a5d4-90fb32e4a6be None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:24.841689 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-877fc9ac-f068-4457-a5d4-90fb32e4a6be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:24.849809 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ae904-7adc-4ce5-abb5-a623a774fe4b admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:24.953183 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 894/1770] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:25.000259 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:25.077713 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c51cd7-9a36-4cb4-adf2-3a998a7f65ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:25.078422 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c51cd7-9a36-4cb4-adf2-3a998a7f65ec admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:25.078691 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c51cd7-9a36-4cb4-adf2-3a998a7f65ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:25.078971 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c51cd7-9a36-4cb4-adf2-3a998a7f65ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:25.079716 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c51cd7-9a36-4cb4-adf2-3a998a7f65ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:25.081613 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c51cd7-9a36-4cb4-adf2-3a998a7f65ec admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:25.104341 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877fc9ac-f068-4457-a5d4-90fb32e4a6be None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:25.106804 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-877fc9ac-f068-4457-a5d4-90fb32e4a6be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:25.208686 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 877/1771] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:03:24 2022] POST /identity/v3/projects => generated 383 bytes in 432 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:25.212231 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c51cd7-9a36-4cb4-adf2-3a998a7f65ec admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:25.316496 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877fc9ac-f068-4457-a5d4-90fb32e4a6be None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:25.402015 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c51cd7-9a36-4cb4-adf2-3a998a7f65ec admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:25.416326 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a8f58e-3c01-4640-a9e7-6bf686849def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:25.417733 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a8f58e-3c01-4640-a9e7-6bf686849def admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:25.418025 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a8f58e-3c01-4640-a9e7-6bf686849def admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:25.418304 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a8f58e-3c01-4640-a9e7-6bf686849def admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:25.420674 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a8f58e-3c01-4640-a9e7-6bf686849def admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:25.426912 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a8f58e-3c01-4640-a9e7-6bf686849def admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:25.523424 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 878/1772] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:03:25 2022] POST /identity/v3/users => generated 292 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:25.526981 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36848b34-1d2c-4038-a3e3-18695f2a72f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:25.527307 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9118545e-dd32-494a-b6e7-77e66d1c9614 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:25.527808 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36848b34-1d2c-4038-a3e3-18695f2a72f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:25.527975 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36848b34-1d2c-4038-a3e3-18695f2a72f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:25.528259 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36848b34-1d2c-4038-a3e3-18695f2a72f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:25.528494 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9118545e-dd32-494a-b6e7-77e66d1c9614 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:25.528793 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9118545e-dd32-494a-b6e7-77e66d1c9614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:25.619901 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36848b34-1d2c-4038-a3e3-18695f2a72f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:25.622455 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36848b34-1d2c-4038-a3e3-18695f2a72f5 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:25.661699 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-9118545e-dd32-494a-b6e7-77e66d1c9614 None None] MFA Rules not processed for user `0e68eff6a08d4193b3167d47f4e1f97c`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:25.669090 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-9118545e-dd32-494a-b6e7-77e66d1c9614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:25.676456 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 895/1773] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 22:03:25.677429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36848b34-1d2c-4038-a3e3-18695f2a72f5 service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:25.726397 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36848b34-1d2c-4038-a3e3-18695f2a72f5 service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:25.740069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a795a31-a9b4-41ec-8e86-4c3bb4695636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:25.740787 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a795a31-a9b4-41ec-8e86-4c3bb4695636 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:25.741067 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a795a31-a9b4-41ec-8e86-4c3bb4695636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:25.741339 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a795a31-a9b4-41ec-8e86-4c3bb4695636 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:25.742188 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a795a31-a9b4-41ec-8e86-4c3bb4695636 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:25.745015 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a795a31-a9b4-41ec-8e86-4c3bb4695636 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:25.751308 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:25.780048 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 896/1774] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:25.788828 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a795a31-a9b4-41ec-8e86-4c3bb4695636 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:25.804125 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a795a31-a9b4-41ec-8e86-4c3bb4695636 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:25.822285 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c43394f-6f52-48a3-95eb-59ace3cc3061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:25.822981 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43394f-6f52-48a3-95eb-59ace3cc3061 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:25.823316 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43394f-6f52-48a3-95eb-59ace3cc3061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:25.824371 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43394f-6f52-48a3-95eb-59ace3cc3061 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:25.824868 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c43394f-6f52-48a3-95eb-59ace3cc3061 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:25.826503 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c43394f-6f52-48a3-95eb-59ace3cc3061 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:25.854333 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 897/1775] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 22:03:25.859013 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c43394f-6f52-48a3-95eb-59ace3cc3061 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:25.884437 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c43394f-6f52-48a3-95eb-59ace3cc3061 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:25.891546 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 879/1776] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:25.901003 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ebfb3b-e786-44ae-a341-2bb1f4cffb18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:25.902451 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ebfb3b-e786-44ae-a341-2bb1f4cffb18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:25.902913 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ebfb3b-e786-44ae-a341-2bb1f4cffb18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:25.903434 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ebfb3b-e786-44ae-a341-2bb1f4cffb18 admin admin] PATH_INFO: `/v3/projects/365637613bba4e26bf79d0e5dc24a2d1/users/e5775e2dab8244edaacb1440323fe111/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:25.953335 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ebfb3b-e786-44ae-a341-2bb1f4cffb18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=365637613bba4e26bf79d0e5dc24a2d1, user_id=e5775e2dab8244edaacb1440323fe111, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:25.956374 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ebfb3b-e786-44ae-a341-2bb1f4cffb18 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:26.002602 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 898/1777] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:03:25 2022] PUT /identity/v3/projects/365637613bba4e26bf79d0e5dc24a2d1/users/e5775e2dab8244edaacb1440323fe111/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:26.014383 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f079d46c-6503-44c0-9d8a-b9a690bd2b7d None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:26.014991 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f079d46c-6503-44c0-9d8a-b9a690bd2b7d None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:26.015479 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f079d46c-6503-44c0-9d8a-b9a690bd2b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:26.041782 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ebfb3b-e786-44ae-a341-2bb1f4cffb18 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:26.044425 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-e4ebfb3b-e786-44ae-a341-2bb1f4cffb18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:26.106357 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-f079d46c-6503-44c0-9d8a-b9a690bd2b7d None None] MFA Rules not processed for user `e5775e2dab8244edaacb1440323fe111`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:26.107664 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-f079d46c-6503-44c0-9d8a-b9a690bd2b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:26.244838 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:26.277988 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ebfb3b-e786-44ae-a341-2bb1f4cffb18 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:26.293907 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58830305-a8bc-4eb3-a2c2-84870a7d1af5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:26.295369 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58830305-a8bc-4eb3-a2c2-84870a7d1af5 service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:26.295369 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58830305-a8bc-4eb3-a2c2-84870a7d1af5 service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:26.295369 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58830305-a8bc-4eb3-a2c2-84870a7d1af5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:26.328618 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 880/1778] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:26.335593 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f079d46c-6503-44c0-9d8a-b9a690bd2b7d None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:26.389035 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58830305-a8bc-4eb3-a2c2-84870a7d1af5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:26.392289 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58830305-a8bc-4eb3-a2c2-84870a7d1af5 service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:26.407205 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 899/1779] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:26 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 22:03:26.443047 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f079d46c-6503-44c0-9d8a-b9a690bd2b7d None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:26.452676 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ed5f6b-86bf-4fbd-934b-86f3b8fe819c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:26.453700 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed5f6b-86bf-4fbd-934b-86f3b8fe819c admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:26.453700 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed5f6b-86bf-4fbd-934b-86f3b8fe819c admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:26.456823 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed5f6b-86bf-4fbd-934b-86f3b8fe819c admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:26.459037 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ed5f6b-86bf-4fbd-934b-86f3b8fe819c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:26.463040 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ed5f6b-86bf-4fbd-934b-86f3b8fe819c admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:26.502945 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 881/1780] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:03:26 2022] POST /identity/v3/projects => generated 385 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:26.505367 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58830305-a8bc-4eb3-a2c2-84870a7d1af5 service nova] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:26.523289 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58830305-a8bc-4eb3-a2c2-84870a7d1af5 service nova] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:26.546338 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ed5f6b-86bf-4fbd-934b-86f3b8fe819c admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:26.550162 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-94ed5f6b-86bf-4fbd-934b-86f3b8fe819c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:26.616146 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a602e7a-d636-4c05-9784-defb5047485f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:26.617489 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a602e7a-d636-4c05-9784-defb5047485f admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:26.617852 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a602e7a-d636-4c05-9784-defb5047485f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:26.618228 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a602e7a-d636-4c05-9784-defb5047485f admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:26.619117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a602e7a-d636-4c05-9784-defb5047485f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:26.623035 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a602e7a-d636-4c05-9784-defb5047485f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:26.631473 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:26.669096 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 900/1781] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:03:26 2022] POST /identity/v3/users => generated 299 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:26.672594 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a602e7a-d636-4c05-9784-defb5047485f admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:26.712745 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a602e7a-d636-4c05-9784-defb5047485f admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:26.717063 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ed5f6b-86bf-4fbd-934b-86f3b8fe819c admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:26.718683 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a149ed22-ae1f-4571-9867-5d2038bab4aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:26.719406 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149ed22-ae1f-4571-9867-5d2038bab4aa admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:26.719711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149ed22-ae1f-4571-9867-5d2038bab4aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:26.720079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149ed22-ae1f-4571-9867-5d2038bab4aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:26.721431 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a149ed22-ae1f-4571-9867-5d2038bab4aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:26.723301 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a149ed22-ae1f-4571-9867-5d2038bab4aa admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:26.734947 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c32b5-eb9b-44bd-9530-1dbaadf35326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:26.734947 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c32b5-eb9b-44bd-9530-1dbaadf35326 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:26.734947 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c32b5-eb9b-44bd-9530-1dbaadf35326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:26.734947 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c32b5-eb9b-44bd-9530-1dbaadf35326 admin admin] PATH_INFO: `/v3/projects/06aee994b46e45ed9ff3d56bfbf49326` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:26.736368 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799c32b5-eb9b-44bd-9530-1dbaadf35326 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06aee994b46e45ed9ff3d56bfbf49326)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:26.740276 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799c32b5-eb9b-44bd-9530-1dbaadf35326 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:26.753422 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 901/1782] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:26.757695 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a149ed22-ae1f-4571-9867-5d2038bab4aa admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:26.788281 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a149ed22-ae1f-4571-9867-5d2038bab4aa admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:26.807561 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b9c11e-3338-4f54-833a-85761a5b19ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:26.810492 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b9c11e-3338-4f54-833a-85761a5b19ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:26.819771 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b9c11e-3338-4f54-833a-85761a5b19ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:26.820802 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b9c11e-3338-4f54-833a-85761a5b19ca admin admin] PATH_INFO: `/v3/projects/4280a5f3e53c41178100e3d90d49aa66/users/8659aafcb2784b13ba5428f0a70be7d6/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:26.865071 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-799c32b5-eb9b-44bd-9530-1dbaadf35326 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06aee994b46e45ed9ff3d56bfbf49326', 'request_id': 'req-799c32b5-eb9b-44bd-9530-1dbaadf35326'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:26.868849 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b9c11e-3338-4f54-833a-85761a5b19ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=4280a5f3e53c41178100e3d90d49aa66, user_id=8659aafcb2784b13ba5428f0a70be7d6, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:26.876666 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b9c11e-3338-4f54-833a-85761a5b19ca admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:26.881634 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-799c32b5-eb9b-44bd-9530-1dbaadf35326 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06aee994b46e45ed9ff3d56bfbf49326', 'request_id': 'req-799c32b5-eb9b-44bd-9530-1dbaadf35326'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:26.894108 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-799c32b5-eb9b-44bd-9530-1dbaadf35326 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06aee994b46e45ed9ff3d56bfbf49326', 'request_id': 'req-799c32b5-eb9b-44bd-9530-1dbaadf35326'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:26.895885 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:26.918288 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 902/1783] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:03:26 2022] PUT /identity/v3/projects/4280a5f3e53c41178100e3d90d49aa66/users/8659aafcb2784b13ba5428f0a70be7d6/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:26.923960 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-799c32b5-eb9b-44bd-9530-1dbaadf35326 admin admin] The token cache is being invalidate because project 06aee994b46e45ed9ff3d56bfbf49326 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:03:26.924459 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-799c32b5-eb9b-44bd-9530-1dbaadf35326 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:26.925223 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b9c11e-3338-4f54-833a-85761a5b19ca admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:26.925694 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-799c32b5-eb9b-44bd-9530-1dbaadf35326 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:26.927425 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-89b9c11e-3338-4f54-833a-85761a5b19ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:26.928689 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 882/1784] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:03:26 2022] DELETE /identity/v3/projects/06aee994b46e45ed9ff3d56bfbf49326 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:26.934512 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995890bf-b6c0-47c3-9119-2771792bc356 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:26.934957 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995890bf-b6c0-47c3-9119-2771792bc356 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:26.935436 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995890bf-b6c0-47c3-9119-2771792bc356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:26.974937 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-995890bf-b6c0-47c3-9119-2771792bc356 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:26.976553 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-995890bf-b6c0-47c3-9119-2771792bc356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:27.112295 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b9c11e-3338-4f54-833a-85761a5b19ca admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:27.119339 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca7b6c5-0bd9-4fd3-a793-597932fa81f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:27.126586 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca7b6c5-0bd9-4fd3-a793-597932fa81f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:27.127189 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca7b6c5-0bd9-4fd3-a793-597932fa81f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:27.127707 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca7b6c5-0bd9-4fd3-a793-597932fa81f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:27.128947 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca7b6c5-0bd9-4fd3-a793-597932fa81f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:27.132201 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca7b6c5-0bd9-4fd3-a793-597932fa81f2 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:27.149347 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 903/1785] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:27.153351 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca7b6c5-0bd9-4fd3-a793-597932fa81f2 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:27.173894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca7b6c5-0bd9-4fd3-a793-597932fa81f2 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:27.176794 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 883/1786] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:27.183854 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febeb4a0-8668-49e8-96b1-baa4bc78dbc2 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:27.184472 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febeb4a0-8668-49e8-96b1-baa4bc78dbc2 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:27.184896 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febeb4a0-8668-49e8-96b1-baa4bc78dbc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:27.185201 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eb58e6-4cb6-411f-b05e-5d996e8e5c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:27.186247 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb58e6-4cb6-411f-b05e-5d996e8e5c7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:27.186634 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb58e6-4cb6-411f-b05e-5d996e8e5c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:27.187073 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb58e6-4cb6-411f-b05e-5d996e8e5c7a admin admin] PATH_INFO: `/v3/projects/4280a5f3e53c41178100e3d90d49aa66/users/8659aafcb2784b13ba5428f0a70be7d6/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:27.191994 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eb58e6-4cb6-411f-b05e-5d996e8e5c7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4280a5f3e53c41178100e3d90d49aa66, user_id=8659aafcb2784b13ba5428f0a70be7d6, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:27.195398 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eb58e6-4cb6-411f-b05e-5d996e8e5c7a admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:27.219127 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-febeb4a0-8668-49e8-96b1-baa4bc78dbc2 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:27.220933 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-febeb4a0-8668-49e8-96b1-baa4bc78dbc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:27.230930 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 904/1787] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:03:27 2022] PUT /identity/v3/projects/4280a5f3e53c41178100e3d90d49aa66/users/8659aafcb2784b13ba5428f0a70be7d6/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:27.233098 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b136fcb1-4795-45f3-9965-72938b563c14 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:27.233525 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b136fcb1-4795-45f3-9965-72938b563c14 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:27.233985 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b136fcb1-4795-45f3-9965-72938b563c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:27.305646 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-b136fcb1-4795-45f3-9965-72938b563c14 None None] MFA Rules not processed for user `8659aafcb2784b13ba5428f0a70be7d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:27.306972 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-b136fcb1-4795-45f3-9965-72938b563c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:27.432112 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 884/1788] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:27.435082 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febeb4a0-8668-49e8-96b1-baa4bc78dbc2 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:27.481650 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febeb4a0-8668-49e8-96b1-baa4bc78dbc2 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:27.513000 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f463cdd-df40-4479-ac29-d5bf95a5ca15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:27.513817 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f463cdd-df40-4479-ac29-d5bf95a5ca15 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:27.514104 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f463cdd-df40-4479-ac29-d5bf95a5ca15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:27.514381 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f463cdd-df40-4479-ac29-d5bf95a5ca15 admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:27.515129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f463cdd-df40-4479-ac29-d5bf95a5ca15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:03:27.519148 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f463cdd-df40-4479-ac29-d5bf95a5ca15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:27.520963 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f463cdd-df40-4479-ac29-d5bf95a5ca15 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:27.561366 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 885/1789] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:03:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:03:27.562969 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f463cdd-df40-4479-ac29-d5bf95a5ca15 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:27.566921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-1f463cdd-df40-4479-ac29-d5bf95a5ca15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:27.606750 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f463cdd-df40-4479-ac29-d5bf95a5ca15 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:27.614701 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f7ee9-d633-481c-9b80-7decf5741148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:27.615809 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f7ee9-d633-481c-9b80-7decf5741148 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:27.616180 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f7ee9-d633-481c-9b80-7decf5741148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:27.616538 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f7ee9-d633-481c-9b80-7decf5741148 admin admin] PATH_INFO: `/v3/projects/10eea15a0d374e6fa9c1cab810111128` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:27.618835 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46f7ee9-d633-481c-9b80-7decf5741148 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10eea15a0d374e6fa9c1cab810111128)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:27.622995 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46f7ee9-d633-481c-9b80-7decf5741148 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:27.631987 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 905/1790] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:27 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 400 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:27.633193 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b136fcb1-4795-45f3-9965-72938b563c14 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:27.659847 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b136fcb1-4795-45f3-9965-72938b563c14 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:27.678841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e1b035-43b9-4e54-9403-b07dd139d129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:27.679938 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e1b035-43b9-4e54-9403-b07dd139d129 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:27.680377 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e1b035-43b9-4e54-9403-b07dd139d129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:27.680865 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e1b035-43b9-4e54-9403-b07dd139d129 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:27.681742 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e1b035-43b9-4e54-9403-b07dd139d129 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:27.684805 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e1b035-43b9-4e54-9403-b07dd139d129 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:27.702585 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-f46f7ee9-d633-481c-9b80-7decf5741148 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10eea15a0d374e6fa9c1cab810111128', 'request_id': 'req-f46f7ee9-d633-481c-9b80-7decf5741148'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:27.715659 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-f46f7ee9-d633-481c-9b80-7decf5741148 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10eea15a0d374e6fa9c1cab810111128', 'request_id': 'req-f46f7ee9-d633-481c-9b80-7decf5741148'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:27.734744 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:27.735938 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-f46f7ee9-d633-481c-9b80-7decf5741148 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10eea15a0d374e6fa9c1cab810111128', 'request_id': 'req-f46f7ee9-d633-481c-9b80-7decf5741148'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:27.757848 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 906/1791] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:03:27 2022] POST /identity/v3/projects => generated 367 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:27.758748 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e1b035-43b9-4e54-9403-b07dd139d129 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:27.762307 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-40e1b035-43b9-4e54-9403-b07dd139d129 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:27.785854 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:27.815669 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-f46f7ee9-d633-481c-9b80-7decf5741148 admin admin] The token cache is being invalidate because project 10eea15a0d374e6fa9c1cab810111128 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:03:27.816302 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-f46f7ee9-d633-481c-9b80-7decf5741148 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:27.817987 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-f46f7ee9-d633-481c-9b80-7decf5741148 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:27.829065 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 886/1792] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:03:27 2022] DELETE /identity/v3/projects/10eea15a0d374e6fa9c1cab810111128 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:27.830129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f7ee9-d633-481c-9b80-7decf5741148 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:27.836269 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-f46f7ee9-d633-481c-9b80-7decf5741148 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:27.967075 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e1b035-43b9-4e54-9403-b07dd139d129 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:27.986318 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc9f57-b6af-4c31-9faf-b2ef08a1dceb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:27.989427 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc9f57-b6af-4c31-9faf-b2ef08a1dceb service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:27.991166 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc9f57-b6af-4c31-9faf-b2ef08a1dceb service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:27.993308 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc9f57-b6af-4c31-9faf-b2ef08a1dceb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:28.008536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-02cc9f57-b6af-4c31-9faf-b2ef08a1dceb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:28.069197 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f7ee9-d633-481c-9b80-7decf5741148 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:28.201051 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cc9f57-b6af-4c31-9faf-b2ef08a1dceb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:28.204827 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cc9f57-b6af-4c31-9faf-b2ef08a1dceb service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:28.243676 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 907/1793] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:03:28.248321 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc9f57-b6af-4c31-9faf-b2ef08a1dceb service cinder] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:28.250763 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-02cc9f57-b6af-4c31-9faf-b2ef08a1dceb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:28.297246 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b616d9d-574b-4459-bd6c-4f667f1ebb75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:28.297956 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b616d9d-574b-4459-bd6c-4f667f1ebb75 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:28.298208 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b616d9d-574b-4459-bd6c-4f667f1ebb75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:28.298486 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b616d9d-574b-4459-bd6c-4f667f1ebb75 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:28.299191 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b616d9d-574b-4459-bd6c-4f667f1ebb75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:28.301152 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b616d9d-574b-4459-bd6c-4f667f1ebb75 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:28.358379 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 887/1794] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:03:27 2022] POST /identity/v3/users => generated 284 bytes in 529 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:28.359311 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b616d9d-574b-4459-bd6c-4f667f1ebb75 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:28.363254 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-1b616d9d-574b-4459-bd6c-4f667f1ebb75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:28.410819 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b616d9d-574b-4459-bd6c-4f667f1ebb75 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:28.439626 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04416373-63da-462c-a994-494e12faec11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:28.439626 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04416373-63da-462c-a994-494e12faec11 service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:28.439626 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04416373-63da-462c-a994-494e12faec11 service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:28.439626 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04416373-63da-462c-a994-494e12faec11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:28.440061 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-04416373-63da-462c-a994-494e12faec11 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:28.484359 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc9f57-b6af-4c31-9faf-b2ef08a1dceb service cinder] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:28.499761 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03db6b0-6767-4be1-a669-66d1a8db8b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:28.501457 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03db6b0-6767-4be1-a669-66d1a8db8b0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:28.501928 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03db6b0-6767-4be1-a669-66d1a8db8b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:28.502357 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03db6b0-6767-4be1-a669-66d1a8db8b0c admin admin] PATH_INFO: `/v3/users/fc36120c63934f6b96dc3aa3a926d3fd` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:28.506277 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03db6b0-6767-4be1-a669-66d1a8db8b0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc36120c63934f6b96dc3aa3a926d3fd)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:28.510264 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03db6b0-6767-4be1-a669-66d1a8db8b0c admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:28.674251 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-e03db6b0-6767-4be1-a669-66d1a8db8b0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc36120c63934f6b96dc3aa3a926d3fd'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:28.693535 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04416373-63da-462c-a994-494e12faec11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:28.700040 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-e03db6b0-6767-4be1-a669-66d1a8db8b0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc36120c63934f6b96dc3aa3a926d3fd'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:28.704827 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04416373-63da-462c-a994-494e12faec11 service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:28.708575 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:28.763422 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 888/1795] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:28 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:03:28.764498 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04416373-63da-462c-a994-494e12faec11 service cinder] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:28.773426 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-e03db6b0-6767-4be1-a669-66d1a8db8b0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc36120c63934f6b96dc3aa3a926d3fd'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:28.774389 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-e03db6b0-6767-4be1-a669-66d1a8db8b0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc36120c63934f6b96dc3aa3a926d3fd'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:28.779137 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-e03db6b0-6767-4be1-a669-66d1a8db8b0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc36120c63934f6b96dc3aa3a926d3fd'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:28.804976 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04416373-63da-462c-a994-494e12faec11 service cinder] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:28.818315 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 908/1796] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:03:28 2022] DELETE /identity/v3/users/fc36120c63934f6b96dc3aa3a926d3fd => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:28.819514 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ef9c61-7673-4f9c-ad54-7a9ce3e9632a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:28.820370 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef9c61-7673-4f9c-ad54-7a9ce3e9632a admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:28.820879 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef9c61-7673-4f9c-ad54-7a9ce3e9632a admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:28.821313 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef9c61-7673-4f9c-ad54-7a9ce3e9632a admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:28.825287 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ef9c61-7673-4f9c-ad54-7a9ce3e9632a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:28.830492 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ef9c61-7673-4f9c-ad54-7a9ce3e9632a admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:28.862132 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03db6b0-6767-4be1-a669-66d1a8db8b0c admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:28.863774 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-e03db6b0-6767-4be1-a669-66d1a8db8b0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:28.865601 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 889/1797] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:29.117853 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03db6b0-6767-4be1-a669-66d1a8db8b0c admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:29.139892 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc8e91e-eabc-4961-a9a3-a63490c7454e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:29.140631 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc8e91e-eabc-4961-a9a3-a63490c7454e admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:29.140900 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc8e91e-eabc-4961-a9a3-a63490c7454e admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:29.143196 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc8e91e-eabc-4961-a9a3-a63490c7454e admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:29.145414 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc8e91e-eabc-4961-a9a3-a63490c7454e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:29.149543 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc8e91e-eabc-4961-a9a3-a63490c7454e admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:29.189593 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ef9c61-7673-4f9c-ad54-7a9ce3e9632a admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:29.200069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 909/1798] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:29.203734 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ef9c61-7673-4f9c-ad54-7a9ce3e9632a admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:29.219077 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca375667-9d07-4dfb-853d-61332d7ac79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:29.221263 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca375667-9d07-4dfb-853d-61332d7ac79a admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:29.221636 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca375667-9d07-4dfb-853d-61332d7ac79a admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:29.221924 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca375667-9d07-4dfb-853d-61332d7ac79a admin admin] PATH_INFO: `/v3/projects/f04e39f503064beb9b217d16e39ba4e8/users/69c10b3164034053af5b2617a215b112/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:29.269207 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca375667-9d07-4dfb-853d-61332d7ac79a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f04e39f503064beb9b217d16e39ba4e8, user_id=69c10b3164034053af5b2617a215b112, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:29.272421 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca375667-9d07-4dfb-853d-61332d7ac79a admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:29.276706 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:29.319358 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 890/1799] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:03:29 2022] PUT /identity/v3/projects/f04e39f503064beb9b217d16e39ba4e8/users/69c10b3164034053af5b2617a215b112/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:29.321167 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f41b1c-416a-4d28-bccf-c025661cc05c None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:29.321167 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f41b1c-416a-4d28-bccf-c025661cc05c None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:29.321724 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f41b1c-416a-4d28-bccf-c025661cc05c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:29.354824 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-50f41b1c-416a-4d28-bccf-c025661cc05c None None] MFA Rules not processed for user `69c10b3164034053af5b2617a215b112`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:29.362334 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-50f41b1c-416a-4d28-bccf-c025661cc05c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:29.463970 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca375667-9d07-4dfb-853d-61332d7ac79a admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:29.465554 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 910/1800] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:29 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:29.587162 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca375667-9d07-4dfb-853d-61332d7ac79a admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:29.597262 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b5e7e5-39ee-44e7-a624-943897936fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:29.598110 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5e7e5-39ee-44e7-a624-943897936fb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:29.598555 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5e7e5-39ee-44e7-a624-943897936fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:29.598947 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5e7e5-39ee-44e7-a624-943897936fb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:29.602176 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b5e7e5-39ee-44e7-a624-943897936fb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:29.604394 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b5e7e5-39ee-44e7-a624-943897936fb3 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:29.697738 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f41b1c-416a-4d28-bccf-c025661cc05c None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:29.710299 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 891/1801] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:03:29 2022] POST /identity/v3/projects => generated 369 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:29.730298 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f41b1c-416a-4d28-bccf-c025661cc05c None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:29.994006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe3b852-29e7-4abd-92d1-61f7b0b5cbec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:29.995037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3b852-29e7-4abd-92d1-61f7b0b5cbec admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:29.996256 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3b852-29e7-4abd-92d1-61f7b0b5cbec admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:29.996740 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3b852-29e7-4abd-92d1-61f7b0b5cbec admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:29.997756 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe3b852-29e7-4abd-92d1-61f7b0b5cbec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:30.009595 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe3b852-29e7-4abd-92d1-61f7b0b5cbec admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:30.026772 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:30.045734 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 911/1802] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:03:29 2022] POST /identity/v3/users => generated 291 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:30.053788 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b5e7e5-39ee-44e7-a624-943897936fb3 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:30.075734 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b5e7e5-39ee-44e7-a624-943897936fb3 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:30.088377 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6e510-bdab-4090-9f59-ec0035f39505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:30.089220 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6e510-bdab-4090-9f59-ec0035f39505 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:30.089575 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6e510-bdab-4090-9f59-ec0035f39505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:30.090013 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6e510-bdab-4090-9f59-ec0035f39505 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:30.107261 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb6e510-bdab-4090-9f59-ec0035f39505 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:30.109353 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb6e510-bdab-4090-9f59-ec0035f39505 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:30.179219 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe3b852-29e7-4abd-92d1-61f7b0b5cbec admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:30.185142 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 892/1803] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:30 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:30.209510 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe3b852-29e7-4abd-92d1-61f7b0b5cbec admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:30.221166 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6237886e-7731-42da-82c9-5667d3f0d190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:30.226463 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237886e-7731-42da-82c9-5667d3f0d190 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:30.227429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237886e-7731-42da-82c9-5667d3f0d190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:30.227946 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237886e-7731-42da-82c9-5667d3f0d190 admin admin] PATH_INFO: `/v3/projects/01f77dac3bb04ec0b7456dc1c88c128e/users/406e28f976f147af8c30f2c85556c296/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:30.295365 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6237886e-7731-42da-82c9-5667d3f0d190 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01f77dac3bb04ec0b7456dc1c88c128e, user_id=406e28f976f147af8c30f2c85556c296, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:30.306922 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6237886e-7731-42da-82c9-5667d3f0d190 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:30.355373 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 912/1804] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:03:30 2022] PUT /identity/v3/projects/01f77dac3bb04ec0b7456dc1c88c128e/users/406e28f976f147af8c30f2c85556c296/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:30.360194 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6e510-bdab-4090-9f59-ec0035f39505 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:30.376189 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6e510-bdab-4090-9f59-ec0035f39505 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:30.551311 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23770e2c-eeb8-4331-b7db-3cb27c6a9f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:30.553673 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23770e2c-eeb8-4331-b7db-3cb27c6a9f45 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:30.554199 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23770e2c-eeb8-4331-b7db-3cb27c6a9f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:30.555583 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23770e2c-eeb8-4331-b7db-3cb27c6a9f45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:30.556588 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23770e2c-eeb8-4331-b7db-3cb27c6a9f45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:30.559501 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23770e2c-eeb8-4331-b7db-3cb27c6a9f45 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:30.591316 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 893/1805] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:30 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:30.607808 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6237886e-7731-42da-82c9-5667d3f0d190 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:30.611177 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23770e2c-eeb8-4331-b7db-3cb27c6a9f45 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:30.613227 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-23770e2c-eeb8-4331-b7db-3cb27c6a9f45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:30.614326 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6237886e-7731-42da-82c9-5667d3f0d190 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:30.633942 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3dfeb5-75aa-42d6-b1ef-861abb97b111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:30.634989 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3dfeb5-75aa-42d6-b1ef-861abb97b111 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:30.635275 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3dfeb5-75aa-42d6-b1ef-861abb97b111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:30.635699 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3dfeb5-75aa-42d6-b1ef-861abb97b111 admin admin] PATH_INFO: `/v3/projects/01f77dac3bb04ec0b7456dc1c88c128e/users/406e28f976f147af8c30f2c85556c296/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:30.640069 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23770e2c-eeb8-4331-b7db-3cb27c6a9f45 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:30.641129 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3dfeb5-75aa-42d6-b1ef-861abb97b111 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01f77dac3bb04ec0b7456dc1c88c128e, user_id=406e28f976f147af8c30f2c85556c296, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:30.645336 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3dfeb5-75aa-42d6-b1ef-861abb97b111 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:30.657178 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1258a5-cec6-4ac0-94a2-8b3aee046cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:30.661082 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1258a5-cec6-4ac0-94a2-8b3aee046cc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:30.661387 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1258a5-cec6-4ac0-94a2-8b3aee046cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:30.661667 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1258a5-cec6-4ac0-94a2-8b3aee046cc4 admin admin] PATH_INFO: `/v3/projects/3ee07fa4e1444a088b1a6650c9c233ca` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:30.667499 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1258a5-cec6-4ac0-94a2-8b3aee046cc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ee07fa4e1444a088b1a6650c9c233ca)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:30.674336 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1258a5-cec6-4ac0-94a2-8b3aee046cc4 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:30.699421 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 913/1806] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:03:30 2022] PUT /identity/v3/projects/01f77dac3bb04ec0b7456dc1c88c128e/users/406e28f976f147af8c30f2c85556c296/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:30.711477 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041607f-f912-4780-a73d-c10492b65cca None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:30.711815 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041607f-f912-4780-a73d-c10492b65cca None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:30.712536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041607f-f912-4780-a73d-c10492b65cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:30.804898 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-4041607f-f912-4780-a73d-c10492b65cca None None] MFA Rules not processed for user `406e28f976f147af8c30f2c85556c296`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:30.808270 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-4041607f-f912-4780-a73d-c10492b65cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:30.848311 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-3c1258a5-cec6-4ac0-94a2-8b3aee046cc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ee07fa4e1444a088b1a6650c9c233ca', 'request_id': 'req-3c1258a5-cec6-4ac0-94a2-8b3aee046cc4'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:30.866467 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-3c1258a5-cec6-4ac0-94a2-8b3aee046cc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ee07fa4e1444a088b1a6650c9c233ca', 'request_id': 'req-3c1258a5-cec6-4ac0-94a2-8b3aee046cc4'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:30.888354 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-3c1258a5-cec6-4ac0-94a2-8b3aee046cc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ee07fa4e1444a088b1a6650c9c233ca', 'request_id': 'req-3c1258a5-cec6-4ac0-94a2-8b3aee046cc4'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:30.890107 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:30.921969 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-3c1258a5-cec6-4ac0-94a2-8b3aee046cc4 admin admin] The token cache is being invalidate because project 3ee07fa4e1444a088b1a6650c9c233ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:03:30.923143 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-3c1258a5-cec6-4ac0-94a2-8b3aee046cc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:30.927689 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-3c1258a5-cec6-4ac0-94a2-8b3aee046cc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:30.930150 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 894/1807] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:03:30 2022] DELETE /identity/v3/projects/3ee07fa4e1444a088b1a6650c9c233ca => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:30.935673 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7dd2e-044f-46dc-b6ad-6311585500d3 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:30.936153 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7dd2e-044f-46dc-b6ad-6311585500d3 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:30.936577 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7dd2e-044f-46dc-b6ad-6311585500d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:30.998708 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-29e7dd2e-044f-46dc-b6ad-6311585500d3 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:30.999997 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-29e7dd2e-044f-46dc-b6ad-6311585500d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:31.039353 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 914/1808] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:30 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:03:31.051506 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4041607f-f912-4780-a73d-c10492b65cca None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:31.061837 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-4041607f-f912-4780-a73d-c10492b65cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:31.189985 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4041607f-f912-4780-a73d-c10492b65cca None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:31.202439 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e17e52-5dd7-4eef-b944-51838ce21c5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:31.204229 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 895/1809] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:31.210499 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e17e52-5dd7-4eef-b944-51838ce21c5f service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:31.210499 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e17e52-5dd7-4eef-b944-51838ce21c5f service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:31.210499 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e17e52-5dd7-4eef-b944-51838ce21c5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:31.212382 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8145f50-1e98-4914-85a8-7a5333e0a93c None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:31.212666 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8145f50-1e98-4914-85a8-7a5333e0a93c None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:31.212948 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8145f50-1e98-4914-85a8-7a5333e0a93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:31.247422 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e17e52-5dd7-4eef-b944-51838ce21c5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:31.256237 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e17e52-5dd7-4eef-b944-51838ce21c5f service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:31.280322 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-d8145f50-1e98-4914-85a8-7a5333e0a93c None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:31.281667 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-d8145f50-1e98-4914-85a8-7a5333e0a93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:31.288222 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 915/1810] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:31 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:03:31.301940 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 896/1811] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:31.303711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e17e52-5dd7-4eef-b944-51838ce21c5f service cinder] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:31.318150 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e17e52-5dd7-4eef-b944-51838ce21c5f service cinder] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:31.324766 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46dc814-430f-4e74-ae9f-0940a2153f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:31.325979 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46dc814-430f-4e74-ae9f-0940a2153f7c admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:31.325979 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46dc814-430f-4e74-ae9f-0940a2153f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:31.326182 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46dc814-430f-4e74-ae9f-0940a2153f7c admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:31.327747 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46dc814-430f-4e74-ae9f-0940a2153f7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:03:31.328228 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46dc814-430f-4e74-ae9f-0940a2153f7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:31.331144 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46dc814-430f-4e74-ae9f-0940a2153f7c admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:31.353091 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 916/1812] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:03:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:03:31.354923 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8145f50-1e98-4914-85a8-7a5333e0a93c None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:31.376384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8145f50-1e98-4914-85a8-7a5333e0a93c None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:31.390893 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0dc8e7-7f8f-4024-80f3-981d1be47c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:31.391782 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0dc8e7-7f8f-4024-80f3-981d1be47c79 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:31.392149 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0dc8e7-7f8f-4024-80f3-981d1be47c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:31.392500 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0dc8e7-7f8f-4024-80f3-981d1be47c79 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:31.393848 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46dc814-430f-4e74-ae9f-0940a2153f7c admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:31.394503 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0dc8e7-7f8f-4024-80f3-981d1be47c79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:31.397548 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0dc8e7-7f8f-4024-80f3-981d1be47c79 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:31.399535 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d46dc814-430f-4e74-ae9f-0940a2153f7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:31.417852 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46dc814-430f-4e74-ae9f-0940a2153f7c admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:31.423773 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3408b5-21a3-46d7-9d6d-70cdc99001ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:31.424743 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3408b5-21a3-46d7-9d6d-70cdc99001ec service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:31.425235 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3408b5-21a3-46d7-9d6d-70cdc99001ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:31.425714 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3408b5-21a3-46d7-9d6d-70cdc99001ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:31.431298 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-ac3408b5-21a3-46d7-9d6d-70cdc99001ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:31.439708 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 897/1813] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:03:31 2022] POST /identity/v3/projects => generated 381 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:03:31.442221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0dc8e7-7f8f-4024-80f3-981d1be47c79 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:31.480324 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:31.546757 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0dc8e7-7f8f-4024-80f3-981d1be47c79 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:31.575483 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaa5ee3-87b6-4d36-a835-ccc31110a914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:31.575623 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa5ee3-87b6-4d36-a835-ccc31110a914 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:31.575795 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa5ee3-87b6-4d36-a835-ccc31110a914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:31.576162 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa5ee3-87b6-4d36-a835-ccc31110a914 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:31.581351 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaa5ee3-87b6-4d36-a835-ccc31110a914 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:31.583359 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaa5ee3-87b6-4d36-a835-ccc31110a914 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:31.632071 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3408b5-21a3-46d7-9d6d-70cdc99001ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:31.638089 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3408b5-21a3-46d7-9d6d-70cdc99001ec service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:31.662954 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 917/1814] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:31 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 22:03:31.664680 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4095cb-ef12-498f-bf6c-b6f0470b0217 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:31.665074 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4095cb-ef12-498f-bf6c-b6f0470b0217 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:31.665358 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4095cb-ef12-498f-bf6c-b6f0470b0217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:31.666655 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 898/1815] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:03:31 2022] POST /identity/v3/users => generated 291 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:31.668062 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaa5ee3-87b6-4d36-a835-ccc31110a914 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:31.681458 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaa5ee3-87b6-4d36-a835-ccc31110a914 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:31.716071 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dabc7e8-fe92-4c18-98e9-6a8a4870bffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:31.717363 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dabc7e8-fe92-4c18-98e9-6a8a4870bffb admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:31.721056 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dabc7e8-fe92-4c18-98e9-6a8a4870bffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:31.721533 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dabc7e8-fe92-4c18-98e9-6a8a4870bffb admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:31.724922 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dabc7e8-fe92-4c18-98e9-6a8a4870bffb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:31.726997 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dabc7e8-fe92-4c18-98e9-6a8a4870bffb admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:31.743583 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-5e4095cb-ef12-498f-bf6c-b6f0470b0217 None None] MFA Rules not processed for user `607b70b4d6e14c95ae2880dd3577c8ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:31.746526 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 899/1816] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:31 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:31.748000 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dabc7e8-fe92-4c18-98e9-6a8a4870bffb admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:31.753214 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-5e4095cb-ef12-498f-bf6c-b6f0470b0217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:31.787277 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dabc7e8-fe92-4c18-98e9-6a8a4870bffb admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:31.796947 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f9e48b-c1c0-4cd1-8d41-172462b72528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:31.798111 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9e48b-c1c0-4cd1-8d41-172462b72528 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:31.799247 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9e48b-c1c0-4cd1-8d41-172462b72528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:31.799853 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9e48b-c1c0-4cd1-8d41-172462b72528 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:31.801170 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f9e48b-c1c0-4cd1-8d41-172462b72528 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:31.803729 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f9e48b-c1c0-4cd1-8d41-172462b72528 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:31.828948 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 900/1817] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:31 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:31.834551 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f9e48b-c1c0-4cd1-8d41-172462b72528 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:31.870717 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f9e48b-c1c0-4cd1-8d41-172462b72528 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:31.889128 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b107907-223c-4b06-a1b1-f32f0d715fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:31.892627 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b107907-223c-4b06-a1b1-f32f0d715fd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:31.894236 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b107907-223c-4b06-a1b1-f32f0d715fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:31.894573 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b107907-223c-4b06-a1b1-f32f0d715fd4 admin admin] PATH_INFO: `/v3/projects/b231665ac65c4bf0b38f6c2eaa862672/users/20ce2fad13fd4cf985f1e9461c400eb2/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:31.931856 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 918/1818] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:31.960452 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b107907-223c-4b06-a1b1-f32f0d715fd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b231665ac65c4bf0b38f6c2eaa862672, user_id=20ce2fad13fd4cf985f1e9461c400eb2, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:31.981532 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b107907-223c-4b06-a1b1-f32f0d715fd4 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:32.025454 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 901/1819] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:03:31 2022] PUT /identity/v3/projects/b231665ac65c4bf0b38f6c2eaa862672/users/20ce2fad13fd4cf985f1e9461c400eb2/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:32.029087 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40c4e7-d986-4b30-83c0-1221b6dc052b None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:32.029423 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40c4e7-d986-4b30-83c0-1221b6dc052b None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:32.029669 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40c4e7-d986-4b30-83c0-1221b6dc052b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:32.088457 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-1b40c4e7-d986-4b30-83c0-1221b6dc052b None None] MFA Rules not processed for user `20ce2fad13fd4cf985f1e9461c400eb2`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:32.091650 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-1b40c4e7-d986-4b30-83c0-1221b6dc052b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:32.193045 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c244bfa-1403-4ffa-a8ad-77a632420559 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:32.193341 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c244bfa-1403-4ffa-a8ad-77a632420559 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:32.193620 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c244bfa-1403-4ffa-a8ad-77a632420559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:32.204487 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:DELETE=2,keystone:SELECT=93,keystone:INSERT=5 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:32.235109 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-9c244bfa-1403-4ffa-a8ad-77a632420559 None None] MFA Rules not processed for user `0e68eff6a08d4193b3167d47f4e1f97c`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:32.238822 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-9c244bfa-1403-4ffa-a8ad-77a632420559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:32.248879 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 919/1820] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:32.249122 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b40c4e7-d986-4b30-83c0-1221b6dc052b None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:32.409970 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 902/1821] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:03:32.480590 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b40c4e7-d986-4b30-83c0-1221b6dc052b None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:32.486544 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90363d73-f689-4ed5-9f01-f8e428386e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:32.487519 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90363d73-f689-4ed5-9f01-f8e428386e42 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:32.488000 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90363d73-f689-4ed5-9f01-f8e428386e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:32.488472 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90363d73-f689-4ed5-9f01-f8e428386e42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:32.489389 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90363d73-f689-4ed5-9f01-f8e428386e42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:32.493159 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90363d73-f689-4ed5-9f01-f8e428386e42 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:32.497643 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:32.536581 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 920/1822] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:03:32 2022] POST /identity/v3/projects => generated 383 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:32.537812 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c244bfa-1403-4ffa-a8ad-77a632420559 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:32.579050 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c244bfa-1403-4ffa-a8ad-77a632420559 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:32.705173 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa45e4a3-7243-48e8-b87e-22adb4296fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:32.706101 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45e4a3-7243-48e8-b87e-22adb4296fae admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:32.706389 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45e4a3-7243-48e8-b87e-22adb4296fae admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:32.707496 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45e4a3-7243-48e8-b87e-22adb4296fae admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:32.708409 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa45e4a3-7243-48e8-b87e-22adb4296fae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:32.710363 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa45e4a3-7243-48e8-b87e-22adb4296fae admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:32.749073 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 903/1823] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:03:32 2022] POST /identity/v3/users => generated 298 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:03:32.752625 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90363d73-f689-4ed5-9f01-f8e428386e42 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:32.768247 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90363d73-f689-4ed5-9f01-f8e428386e42 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:32.778330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d4fda4-321b-4430-9dda-bb0fd31e554e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:32.779023 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4fda4-321b-4430-9dda-bb0fd31e554e admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:32.779303 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4fda4-321b-4430-9dda-bb0fd31e554e admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:32.779614 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4fda4-321b-4430-9dda-bb0fd31e554e admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:32.780381 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d4fda4-321b-4430-9dda-bb0fd31e554e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:32.782259 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d4fda4-321b-4430-9dda-bb0fd31e554e admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:32.795090 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 921/1824] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:32 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:32.798866 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa45e4a3-7243-48e8-b87e-22adb4296fae admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:32.811979 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa45e4a3-7243-48e8-b87e-22adb4296fae admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:32.817638 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d827b66e-3db2-4fe7-83b2-87ea3c5b0a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:32.818646 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827b66e-3db2-4fe7-83b2-87ea3c5b0a20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:32.819018 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827b66e-3db2-4fe7-83b2-87ea3c5b0a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:32.819298 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827b66e-3db2-4fe7-83b2-87ea3c5b0a20 admin admin] PATH_INFO: `/v3/projects/545ce39f2a324c4786ffc887f1a1b630/users/d39df364b9c245e8bfc55d4e66414aa6/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:32.841065 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d827b66e-3db2-4fe7-83b2-87ea3c5b0a20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=545ce39f2a324c4786ffc887f1a1b630, user_id=d39df364b9c245e8bfc55d4e66414aa6, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:32.846813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d827b66e-3db2-4fe7-83b2-87ea3c5b0a20 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:32.874041 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 904/1825] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:03:32 2022] PUT /identity/v3/projects/545ce39f2a324c4786ffc887f1a1b630/users/d39df364b9c245e8bfc55d4e66414aa6/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:32.876818 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d4fda4-321b-4430-9dda-bb0fd31e554e admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:32.890029 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d4fda4-321b-4430-9dda-bb0fd31e554e admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:32.981157 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65451454-9420-499b-8a1a-86502c5040a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:32.981992 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65451454-9420-499b-8a1a-86502c5040a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:32.982280 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65451454-9420-499b-8a1a-86502c5040a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:32.982556 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65451454-9420-499b-8a1a-86502c5040a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:32.983287 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65451454-9420-499b-8a1a-86502c5040a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:32.984994 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65451454-9420-499b-8a1a-86502c5040a5 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:33.001638 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 922/1826] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:32 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:33.005546 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d827b66e-3db2-4fe7-83b2-87ea3c5b0a20 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:33.046799 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d827b66e-3db2-4fe7-83b2-87ea3c5b0a20 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:33.061818 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7534e5ec-2945-4cec-b132-911c40162c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:33.062887 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534e5ec-2945-4cec-b132-911c40162c6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:33.068010 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534e5ec-2945-4cec-b132-911c40162c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:33.068374 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534e5ec-2945-4cec-b132-911c40162c6d admin admin] PATH_INFO: `/v3/projects/545ce39f2a324c4786ffc887f1a1b630/users/d39df364b9c245e8bfc55d4e66414aa6/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:33.081399 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7534e5ec-2945-4cec-b132-911c40162c6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=545ce39f2a324c4786ffc887f1a1b630, user_id=d39df364b9c245e8bfc55d4e66414aa6, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:33.084598 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7534e5ec-2945-4cec-b132-911c40162c6d admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:33.116834 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 905/1827] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:03:33 2022] PUT /identity/v3/projects/545ce39f2a324c4786ffc887f1a1b630/users/d39df364b9c245e8bfc55d4e66414aa6/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:33.118786 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5d304-de52-40dc-b983-75c9dd3f7da7 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:33.119108 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5d304-de52-40dc-b983-75c9dd3f7da7 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:33.119412 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5d304-de52-40dc-b983-75c9dd3f7da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:33.182850 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-a7d5d304-de52-40dc-b983-75c9dd3f7da7 None None] MFA Rules not processed for user `d39df364b9c245e8bfc55d4e66414aa6`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:33.184302 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a7d5d304-de52-40dc-b983-75c9dd3f7da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:33.264132 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376dfd2-817d-4f5d-a30c-0fca1e0c1eb4 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:33.264485 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376dfd2-817d-4f5d-a30c-0fca1e0c1eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:33.264777 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376dfd2-817d-4f5d-a30c-0fca1e0c1eb4 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:33.267370 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 906/1828] 10.222.0.13 () {54 vars in 912 bytes} [Tue Jun 7 22:03:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 22:03:33.275085 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab20247-2c83-4aab-a48b-6e9e6530c894 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:33.275524 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab20247-2c83-4aab-a48b-6e9e6530c894 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:33.275783 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab20247-2c83-4aab-a48b-6e9e6530c894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:33.296020 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 923/1829] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:33 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:33.297202 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-7ab20247-2c83-4aab-a48b-6e9e6530c894 None None] MFA Rules not processed for user `10242251e6284217bfb79c67b1567f75`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:33.298567 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-7ab20247-2c83-4aab-a48b-6e9e6530c894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:33.307288 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d5d304-de52-40dc-b983-75c9dd3f7da7 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:33.380320 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 907/1830] 10.222.0.13 () {58 vars in 1009 bytes} [Tue Jun 7 22:03:33 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:33.380644 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d5d304-de52-40dc-b983-75c9dd3f7da7 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:33.384948 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b902a93-5efc-4810-a120-9a6790fcf707 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:33.385704 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b902a93-5efc-4810-a120-9a6790fcf707 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:33.385997 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b902a93-5efc-4810-a120-9a6790fcf707 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:33.386444 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b902a93-5efc-4810-a120-9a6790fcf707 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:33.393106 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab20247-2c83-4aab-a48b-6e9e6530c894 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:33.395129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-7ab20247-2c83-4aab-a48b-6e9e6530c894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:33.400478 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b902a93-5efc-4810-a120-9a6790fcf707 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:33.403201 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b902a93-5efc-4810-a120-9a6790fcf707 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:33.408302 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:33.409843 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab20247-2c83-4aab-a48b-6e9e6530c894 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:33.414779 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c45508-79a9-40bf-8ae9-05d902f2800b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:33.415472 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c45508-79a9-40bf-8ae9-05d902f2800b service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:33.415764 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c45508-79a9-40bf-8ae9-05d902f2800b service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:33.416044 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c45508-79a9-40bf-8ae9-05d902f2800b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:33.425309 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 924/1831] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:33 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:03:33.433262 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c45508-79a9-40bf-8ae9-05d902f2800b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:33.434732 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c45508-79a9-40bf-8ae9-05d902f2800b service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:33.443552 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:33.450593 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 908/1832] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 22:03:33.532508 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b902a93-5efc-4810-a120-9a6790fcf707 service cinder] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:33.544742 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b902a93-5efc-4810-a120-9a6790fcf707 service cinder] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:33.551458 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9febf54e-195a-4ec3-8438-f3561c13f270 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:33.552122 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9febf54e-195a-4ec3-8438-f3561c13f270 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:33.552410 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9febf54e-195a-4ec3-8438-f3561c13f270 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:33.552683 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9febf54e-195a-4ec3-8438-f3561c13f270 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:33.631063 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9febf54e-195a-4ec3-8438-f3561c13f270 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:33.633164 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9febf54e-195a-4ec3-8438-f3561c13f270 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:33.645678 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 925/1833] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:33 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:03:35.838015 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c45508-79a9-40bf-8ae9-05d902f2800b service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:35.839712 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-70c45508-79a9-40bf-8ae9-05d902f2800b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:35.916507 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c45508-79a9-40bf-8ae9-05d902f2800b service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:35.921806 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d632b-221e-48f2-a5cc-116743e8d087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:35.922700 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d632b-221e-48f2-a5cc-116743e8d087 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:35.923048 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d632b-221e-48f2-a5cc-116743e8d087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:35.923365 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d632b-221e-48f2-a5cc-116743e8d087 admin admin] PATH_INFO: `/v3/users/69c10b3164034053af5b2617a215b112` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:35.924622 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536d632b-221e-48f2-a5cc-116743e8d087 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69c10b3164034053af5b2617a215b112)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:35.926591 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536d632b-221e-48f2-a5cc-116743e8d087 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:36.048967 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-536d632b-221e-48f2-a5cc-116743e8d087 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69c10b3164034053af5b2617a215b112'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:36.056786 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-536d632b-221e-48f2-a5cc-116743e8d087 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69c10b3164034053af5b2617a215b112'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:36.131083 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-536d632b-221e-48f2-a5cc-116743e8d087 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69c10b3164034053af5b2617a215b112'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:36.160571 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-536d632b-221e-48f2-a5cc-116743e8d087 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69c10b3164034053af5b2617a215b112'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:36.182003 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-536d632b-221e-48f2-a5cc-116743e8d087 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69c10b3164034053af5b2617a215b112'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:36.206064 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 909/1834] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:03:35 2022] DELETE /identity/v3/users/69c10b3164034053af5b2617a215b112 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:36.211299 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8738996-82c0-429d-bfe3-baf68e7e7f33 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:36.212264 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8738996-82c0-429d-bfe3-baf68e7e7f33 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:36.212716 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8738996-82c0-429d-bfe3-baf68e7e7f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:36.279834 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-d8738996-82c0-429d-bfe3-baf68e7e7f33 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:36.281563 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d8738996-82c0-429d-bfe3-baf68e7e7f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:36.417496 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 926/1835] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:36.429184 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d632b-221e-48f2-a5cc-116743e8d087 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:36.430643 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-536d632b-221e-48f2-a5cc-116743e8d087 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:36.544180 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d632b-221e-48f2-a5cc-116743e8d087 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:36.548442 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17a359-c94f-4e4f-adac-c0f8a5b16ff2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:36.549108 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17a359-c94f-4e4f-adac-c0f8a5b16ff2 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:36.549475 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17a359-c94f-4e4f-adac-c0f8a5b16ff2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:36.549756 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17a359-c94f-4e4f-adac-c0f8a5b16ff2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:36.570028 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c17a359-c94f-4e4f-adac-c0f8a5b16ff2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:36.571901 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c17a359-c94f-4e4f-adac-c0f8a5b16ff2 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:36.587018 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 910/1836] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 22:03:36.992332 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8738996-82c0-429d-bfe3-baf68e7e7f33 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:36.994219 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d8738996-82c0-429d-bfe3-baf68e7e7f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:37.012260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8738996-82c0-429d-bfe3-baf68e7e7f33 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:37.028483 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa79281-5aa2-4c8f-aabd-e66480495d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:37.029492 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa79281-5aa2-4c8f-aabd-e66480495d59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:37.029808 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa79281-5aa2-4c8f-aabd-e66480495d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:37.030132 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa79281-5aa2-4c8f-aabd-e66480495d59 admin admin] PATH_INFO: `/v3/projects/f04e39f503064beb9b217d16e39ba4e8` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:37.031526 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa79281-5aa2-4c8f-aabd-e66480495d59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f04e39f503064beb9b217d16e39ba4e8)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:37.033855 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa79281-5aa2-4c8f-aabd-e66480495d59 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:37.095980 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-4aa79281-5aa2-4c8f-aabd-e66480495d59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f04e39f503064beb9b217d16e39ba4e8', 'request_id': 'req-4aa79281-5aa2-4c8f-aabd-e66480495d59'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:37.104108 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-4aa79281-5aa2-4c8f-aabd-e66480495d59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f04e39f503064beb9b217d16e39ba4e8', 'request_id': 'req-4aa79281-5aa2-4c8f-aabd-e66480495d59'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:37.109795 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-4aa79281-5aa2-4c8f-aabd-e66480495d59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f04e39f503064beb9b217d16e39ba4e8', 'request_id': 'req-4aa79281-5aa2-4c8f-aabd-e66480495d59'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:37.123221 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-4aa79281-5aa2-4c8f-aabd-e66480495d59 admin admin] The token cache is being invalidate because project f04e39f503064beb9b217d16e39ba4e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:03:37.123560 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-4aa79281-5aa2-4c8f-aabd-e66480495d59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:37.124210 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-4aa79281-5aa2-4c8f-aabd-e66480495d59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:37.125819 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 927/1837] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:03:36 2022] DELETE /identity/v3/projects/f04e39f503064beb9b217d16e39ba4e8 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:37.129947 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17a359-c94f-4e4f-adac-c0f8a5b16ff2 service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:37.141641 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17a359-c94f-4e4f-adac-c0f8a5b16ff2 service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:37.160514 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:37.211130 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663f9d24-a665-43f4-bbe4-e4684fded179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:37.212042 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f9d24-a665-43f4-bbe4-e4684fded179 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:37.212326 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f9d24-a665-43f4-bbe4-e4684fded179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:37.212607 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f9d24-a665-43f4-bbe4-e4684fded179 admin admin] PATH_INFO: `/v3/users/406e28f976f147af8c30f2c85556c296` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:37.214519 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663f9d24-a665-43f4-bbe4-e4684fded179 admin admin] RBAC: Authorizing `identity:delete_user(user_id=406e28f976f147af8c30f2c85556c296)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:37.216656 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663f9d24-a665-43f4-bbe4-e4684fded179 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:37.281666 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-663f9d24-a665-43f4-bbe4-e4684fded179 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '406e28f976f147af8c30f2c85556c296'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:37.282256 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-663f9d24-a665-43f4-bbe4-e4684fded179 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '406e28f976f147af8c30f2c85556c296'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:37.291487 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-663f9d24-a665-43f4-bbe4-e4684fded179 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '406e28f976f147af8c30f2c85556c296'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:37.305251 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-663f9d24-a665-43f4-bbe4-e4684fded179 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '406e28f976f147af8c30f2c85556c296'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:37.319761 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-663f9d24-a665-43f4-bbe4-e4684fded179 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '406e28f976f147af8c30f2c85556c296'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:37.337371 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 911/1838] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:03:37 2022] DELETE /identity/v3/users/406e28f976f147af8c30f2c85556c296 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:03:37.780127 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa79281-5aa2-4c8f-aabd-e66480495d59 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:37.781577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-4aa79281-5aa2-4c8f-aabd-e66480495d59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:37.796668 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa79281-5aa2-4c8f-aabd-e66480495d59 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:37.820912 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:37.869486 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb669dd7-8738-4e5d-8c25-5930c15a1e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:37.871023 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb669dd7-8738-4e5d-8c25-5930c15a1e56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:37.871306 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb669dd7-8738-4e5d-8c25-5930c15a1e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:37.871611 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb669dd7-8738-4e5d-8c25-5930c15a1e56 admin admin] PATH_INFO: `/v3/projects/01f77dac3bb04ec0b7456dc1c88c128e` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:37.872798 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb669dd7-8738-4e5d-8c25-5930c15a1e56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01f77dac3bb04ec0b7456dc1c88c128e)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:37.874742 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb669dd7-8738-4e5d-8c25-5930c15a1e56 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:37.933595 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-eb669dd7-8738-4e5d-8c25-5930c15a1e56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01f77dac3bb04ec0b7456dc1c88c128e', 'request_id': 'req-eb669dd7-8738-4e5d-8c25-5930c15a1e56'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:37.941680 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-eb669dd7-8738-4e5d-8c25-5930c15a1e56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01f77dac3bb04ec0b7456dc1c88c128e', 'request_id': 'req-eb669dd7-8738-4e5d-8c25-5930c15a1e56'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:37.948297 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-eb669dd7-8738-4e5d-8c25-5930c15a1e56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01f77dac3bb04ec0b7456dc1c88c128e', 'request_id': 'req-eb669dd7-8738-4e5d-8c25-5930c15a1e56'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:37.962685 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-eb669dd7-8738-4e5d-8c25-5930c15a1e56 admin admin] The token cache is being invalidate because project 01f77dac3bb04ec0b7456dc1c88c128e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:03:37.962977 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-eb669dd7-8738-4e5d-8c25-5930c15a1e56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:37.964466 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-eb669dd7-8738-4e5d-8c25-5930c15a1e56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:37.967663 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 928/1839] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:03:37 2022] DELETE /identity/v3/projects/01f77dac3bb04ec0b7456dc1c88c128e => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:03:37.972572 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a2850-a90b-4e0b-9d3d-e0a0371a2119 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:37.972893 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a2850-a90b-4e0b-9d3d-e0a0371a2119 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:37.973212 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a2850-a90b-4e0b-9d3d-e0a0371a2119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:37.993822 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-157a2850-a90b-4e0b-9d3d-e0a0371a2119 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:37.995177 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-157a2850-a90b-4e0b-9d3d-e0a0371a2119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:38.067250 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 912/1840] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:38.077241 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2685d82-92d2-44ea-b0e8-7ebff22644e0 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:38.077856 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2685d82-92d2-44ea-b0e8-7ebff22644e0 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:38.078366 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2685d82-92d2-44ea-b0e8-7ebff22644e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:38.098042 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-c2685d82-92d2-44ea-b0e8-7ebff22644e0 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:38.099237 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c2685d82-92d2-44ea-b0e8-7ebff22644e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:38.107923 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 929/1841] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:38.112169 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a2850-a90b-4e0b-9d3d-e0a0371a2119 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:38.122117 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a2850-a90b-4e0b-9d3d-e0a0371a2119 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:38.127753 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994b4fc9-ade3-46c1-a79d-907496325dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:38.128408 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b4fc9-ade3-46c1-a79d-907496325dc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:38.128694 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b4fc9-ade3-46c1-a79d-907496325dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:38.128968 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b4fc9-ade3-46c1-a79d-907496325dc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:38.129692 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994b4fc9-ade3-46c1-a79d-907496325dc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:03:38.129997 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994b4fc9-ade3-46c1-a79d-907496325dc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:38.131776 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994b4fc9-ade3-46c1-a79d-907496325dc8 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:38.145676 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 913/1842] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:03:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:03:38.149904 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2685d82-92d2-44ea-b0e8-7ebff22644e0 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:38.159883 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2685d82-92d2-44ea-b0e8-7ebff22644e0 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:38.165592 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c19ac-834c-4776-8087-78972096e079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:38.166225 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c19ac-834c-4776-8087-78972096e079 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:38.166515 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c19ac-834c-4776-8087-78972096e079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:38.167620 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c19ac-834c-4776-8087-78972096e079 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:38.168198 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849c19ac-834c-4776-8087-78972096e079 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:38.171059 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849c19ac-834c-4776-8087-78972096e079 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:38.202282 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 930/1843] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:03:38 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:03:38.206335 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994b4fc9-ade3-46c1-a79d-907496325dc8 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:38.217104 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994b4fc9-ade3-46c1-a79d-907496325dc8 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:38.245758 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:38.283673 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e620750b-4777-4297-a3f8-2d1d870bc734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:38.284477 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e620750b-4777-4297-a3f8-2d1d870bc734 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:38.284972 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e620750b-4777-4297-a3f8-2d1d870bc734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:38.285334 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e620750b-4777-4297-a3f8-2d1d870bc734 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:38.286065 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e620750b-4777-4297-a3f8-2d1d870bc734 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:38.288058 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e620750b-4777-4297-a3f8-2d1d870bc734 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:38.313275 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 914/1844] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:03:38 2022] POST /identity/v3/users => generated 286 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:03:38.317762 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c19ac-834c-4776-8087-78972096e079 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:38.328128 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c19ac-834c-4776-8087-78972096e079 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:38.333352 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3608db63-fa85-44e7-844e-bb739e27359d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:38.334034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3608db63-fa85-44e7-844e-bb739e27359d admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:38.334320 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3608db63-fa85-44e7-844e-bb739e27359d admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:38.334600 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3608db63-fa85-44e7-844e-bb739e27359d admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:38.335500 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3608db63-fa85-44e7-844e-bb739e27359d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:38.337101 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3608db63-fa85-44e7-844e-bb739e27359d admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:38.349346 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 931/1845] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:38.353642 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e620750b-4777-4297-a3f8-2d1d870bc734 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:38.364617 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e620750b-4777-4297-a3f8-2d1d870bc734 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:38.370329 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10843c3c-4d63-40a3-9f50-6a6141492cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:38.371096 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10843c3c-4d63-40a3-9f50-6a6141492cc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:38.371492 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10843c3c-4d63-40a3-9f50-6a6141492cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:38.371846 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10843c3c-4d63-40a3-9f50-6a6141492cc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:38.372652 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10843c3c-4d63-40a3-9f50-6a6141492cc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:38.374361 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10843c3c-4d63-40a3-9f50-6a6141492cc3 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:38.393898 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 915/1846] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:38 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 22:03:38.395067 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3608db63-fa85-44e7-844e-bb739e27359d admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:38.406134 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3608db63-fa85-44e7-844e-bb739e27359d admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:38.411673 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf92ee3-7654-4e3c-b35c-46e08f1e20f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:38.412553 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf92ee3-7654-4e3c-b35c-46e08f1e20f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:38.412833 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf92ee3-7654-4e3c-b35c-46e08f1e20f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:38.413102 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf92ee3-7654-4e3c-b35c-46e08f1e20f5 admin admin] PATH_INFO: `/v3/projects/e224d637e96d4c168ac90b79697bb89b/users/2eba784f64284eb1b1621c432e29ae3f/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:38.428939 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf92ee3-7654-4e3c-b35c-46e08f1e20f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e224d637e96d4c168ac90b79697bb89b, user_id=2eba784f64284eb1b1621c432e29ae3f, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:38.431728 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf92ee3-7654-4e3c-b35c-46e08f1e20f5 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:38.445446 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 932/1847] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:03:38 2022] PUT /identity/v3/projects/e224d637e96d4c168ac90b79697bb89b/users/2eba784f64284eb1b1621c432e29ae3f/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:38.451150 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0202db4-7433-4067-9a60-54754c13fd30 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:38.451472 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0202db4-7433-4067-9a60-54754c13fd30 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:38.451759 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0202db4-7433-4067-9a60-54754c13fd30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:38.471430 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-b0202db4-7433-4067-9a60-54754c13fd30 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:38.472739 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-b0202db4-7433-4067-9a60-54754c13fd30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:38.580298 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 916/1848] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:38.625514 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf92ee3-7654-4e3c-b35c-46e08f1e20f5 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:38.627481 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-faf92ee3-7654-4e3c-b35c-46e08f1e20f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:38.722496 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:38.732570 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf92ee3-7654-4e3c-b35c-46e08f1e20f5 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:38.738700 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd204a8d-3456-4654-9899-e8472b9c9716 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:38.739699 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd204a8d-3456-4654-9899-e8472b9c9716 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:38.740128 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd204a8d-3456-4654-9899-e8472b9c9716 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:38.740715 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd204a8d-3456-4654-9899-e8472b9c9716 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:38.756648 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd204a8d-3456-4654-9899-e8472b9c9716 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:38.761159 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd204a8d-3456-4654-9899-e8472b9c9716 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:38.788945 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 933/1849] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:03:39.452719 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0202db4-7433-4067-9a60-54754c13fd30 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:39.454948 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-b0202db4-7433-4067-9a60-54754c13fd30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:39.539888 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0202db4-7433-4067-9a60-54754c13fd30 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:39.546804 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86f0f2c-fb8e-4512-8cf5-853d0f09cbfe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:39.547852 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f0f2c-fb8e-4512-8cf5-853d0f09cbfe service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:39.548137 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f0f2c-fb8e-4512-8cf5-853d0f09cbfe service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:39.548412 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f0f2c-fb8e-4512-8cf5-853d0f09cbfe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:39.550996 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-d86f0f2c-fb8e-4512-8cf5-853d0f09cbfe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:39.629352 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:39.634723 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86f0f2c-fb8e-4512-8cf5-853d0f09cbfe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:39.639397 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86f0f2c-fb8e-4512-8cf5-853d0f09cbfe service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:39.657559 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd204a8d-3456-4654-9899-e8472b9c9716 service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:39.659396 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-cd204a8d-3456-4654-9899-e8472b9c9716 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:39.659797 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 917/1850] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:39 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:03:39.707147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd204a8d-3456-4654-9899-e8472b9c9716 service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:39.712559 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd80248-7f1d-4f2a-bf8e-865ff07851f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:39.714121 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd80248-7f1d-4f2a-bf8e-865ff07851f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:39.714472 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd80248-7f1d-4f2a-bf8e-865ff07851f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:39.714826 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd80248-7f1d-4f2a-bf8e-865ff07851f5 admin admin] PATH_INFO: `/v3/users/20ce2fad13fd4cf985f1e9461c400eb2` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:39.716499 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd80248-7f1d-4f2a-bf8e-865ff07851f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20ce2fad13fd4cf985f1e9461c400eb2)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:39.718892 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd80248-7f1d-4f2a-bf8e-865ff07851f5 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:39.829084 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8dd80248-7f1d-4f2a-bf8e-865ff07851f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20ce2fad13fd4cf985f1e9461c400eb2'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:39.840931 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8dd80248-7f1d-4f2a-bf8e-865ff07851f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20ce2fad13fd4cf985f1e9461c400eb2'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:39.859552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8dd80248-7f1d-4f2a-bf8e-865ff07851f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20ce2fad13fd4cf985f1e9461c400eb2'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:39.860381 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8dd80248-7f1d-4f2a-bf8e-865ff07851f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20ce2fad13fd4cf985f1e9461c400eb2'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:39.861001 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8dd80248-7f1d-4f2a-bf8e-865ff07851f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20ce2fad13fd4cf985f1e9461c400eb2'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:39.891456 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 934/1851] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:03:39 2022] DELETE /identity/v3/users/20ce2fad13fd4cf985f1e9461c400eb2 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:39.893417 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10798d6-62d1-4316-8dcc-4bb0f5f567e3 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:39.893818 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10798d6-62d1-4316-8dcc-4bb0f5f567e3 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:39.894242 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10798d6-62d1-4316-8dcc-4bb0f5f567e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:39.936696 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-e10798d6-62d1-4316-8dcc-4bb0f5f567e3 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:39.938075 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-e10798d6-62d1-4316-8dcc-4bb0f5f567e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:40.060847 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 918/1852] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:40.070628 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd80248-7f1d-4f2a-bf8e-865ff07851f5 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:40.072250 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-8dd80248-7f1d-4f2a-bf8e-865ff07851f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:40.171895 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd80248-7f1d-4f2a-bf8e-865ff07851f5 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:40.178290 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edca29ae-e85e-40f5-bd90-c37ade661705 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:40.179999 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca29ae-e85e-40f5-bd90-c37ade661705 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:40.181044 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca29ae-e85e-40f5-bd90-c37ade661705 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:40.181608 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca29ae-e85e-40f5-bd90-c37ade661705 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:40.200868 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edca29ae-e85e-40f5-bd90-c37ade661705 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:40.202431 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edca29ae-e85e-40f5-bd90-c37ade661705 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:40.228672 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 935/1853] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:03:40.444329 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abfdcae-a43c-483e-bd21-9c2cdfa36e38 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:40.444624 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abfdcae-a43c-483e-bd21-9c2cdfa36e38 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:40.444922 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abfdcae-a43c-483e-bd21-9c2cdfa36e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:40.476426 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-8abfdcae-a43c-483e-bd21-9c2cdfa36e38 None None] MFA Rules not processed for user `2eba784f64284eb1b1621c432e29ae3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:40.477663 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-8abfdcae-a43c-483e-bd21-9c2cdfa36e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:40.567965 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 919/1854] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:40.572672 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edca29ae-e85e-40f5-bd90-c37ade661705 service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:40.575298 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-edca29ae-e85e-40f5-bd90-c37ade661705 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:40.593672 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edca29ae-e85e-40f5-bd90-c37ade661705 service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:40.600408 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea68864a-7995-4dd9-9dd9-896cdc3f5909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:40.602031 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68864a-7995-4dd9-9dd9-896cdc3f5909 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:40.602162 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68864a-7995-4dd9-9dd9-896cdc3f5909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:40.602349 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68864a-7995-4dd9-9dd9-896cdc3f5909 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:40.603034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea68864a-7995-4dd9-9dd9-896cdc3f5909 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:40.605653 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea68864a-7995-4dd9-9dd9-896cdc3f5909 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:40.640264 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 936/1855] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:03:40 2022] POST /identity/v3/projects => generated 371 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:40.644208 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abfdcae-a43c-483e-bd21-9c2cdfa36e38 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:40.659369 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abfdcae-a43c-483e-bd21-9c2cdfa36e38 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:40.732505 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279e70ab-245a-404a-955a-c3356ca2135c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:40.733439 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279e70ab-245a-404a-955a-c3356ca2135c admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:40.734174 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279e70ab-245a-404a-955a-c3356ca2135c admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:40.734908 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279e70ab-245a-404a-955a-c3356ca2135c admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:40.735948 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279e70ab-245a-404a-955a-c3356ca2135c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:40.738478 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279e70ab-245a-404a-955a-c3356ca2135c admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:40.778533 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 920/1856] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:03:40 2022] POST /identity/v3/users => generated 292 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:03:40.782873 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea68864a-7995-4dd9-9dd9-896cdc3f5909 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:40.785032 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-ea68864a-7995-4dd9-9dd9-896cdc3f5909 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:40.789770 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279e70ab-245a-404a-955a-c3356ca2135c admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:40.809375 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea68864a-7995-4dd9-9dd9-896cdc3f5909 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:40.816437 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279e70ab-245a-404a-955a-c3356ca2135c admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:40.820030 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9003ce94-7d24-4f5f-81dd-b2a943575a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:40.820918 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9003ce94-7d24-4f5f-81dd-b2a943575a21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:40.821221 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9003ce94-7d24-4f5f-81dd-b2a943575a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:40.821459 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9003ce94-7d24-4f5f-81dd-b2a943575a21 admin admin] PATH_INFO: `/v3/projects/b231665ac65c4bf0b38f6c2eaa862672` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:40.826211 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9003ce94-7d24-4f5f-81dd-b2a943575a21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b231665ac65c4bf0b38f6c2eaa862672)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:40.828817 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9003ce94-7d24-4f5f-81dd-b2a943575a21 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:40.833577 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d8a1ad-67f8-4b35-8925-1b73578c3d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:40.834204 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8a1ad-67f8-4b35-8925-1b73578c3d2d admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:40.834482 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8a1ad-67f8-4b35-8925-1b73578c3d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:40.834754 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8a1ad-67f8-4b35-8925-1b73578c3d2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:40.835529 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d8a1ad-67f8-4b35-8925-1b73578c3d2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:40.837775 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d8a1ad-67f8-4b35-8925-1b73578c3d2d admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:40.855522 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 921/1857] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:40 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:40.856493 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d8a1ad-67f8-4b35-8925-1b73578c3d2d admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:40.863999 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:40.868942 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d8a1ad-67f8-4b35-8925-1b73578c3d2d admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:40.878266 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e92b2a-a6ac-4707-93a9-08b958f3345d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:40.879277 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e92b2a-a6ac-4707-93a9-08b958f3345d admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:40.879680 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e92b2a-a6ac-4707-93a9-08b958f3345d admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:40.880024 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e92b2a-a6ac-4707-93a9-08b958f3345d admin admin] PATH_INFO: `/v3/projects/0fd0d297527b4701bb31615d46055539/users/18b1f026a28e42dd81a741d024f1d4fb/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:40.897804 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e92b2a-a6ac-4707-93a9-08b958f3345d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fd0d297527b4701bb31615d46055539, user_id=18b1f026a28e42dd81a741d024f1d4fb, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:40.898939 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:40.900612 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e92b2a-a6ac-4707-93a9-08b958f3345d admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:40.912926 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-9003ce94-7d24-4f5f-81dd-b2a943575a21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b231665ac65c4bf0b38f6c2eaa862672', 'request_id': 'req-9003ce94-7d24-4f5f-81dd-b2a943575a21'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:40.918841 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 922/1858] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:03:40 2022] PUT /identity/v3/projects/0fd0d297527b4701bb31615d46055539/users/18b1f026a28e42dd81a741d024f1d4fb/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:40.923072 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-9003ce94-7d24-4f5f-81dd-b2a943575a21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b231665ac65c4bf0b38f6c2eaa862672', 'request_id': 'req-9003ce94-7d24-4f5f-81dd-b2a943575a21'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:40.923219 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e92b2a-a6ac-4707-93a9-08b958f3345d admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:40.933750 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-9003ce94-7d24-4f5f-81dd-b2a943575a21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b231665ac65c4bf0b38f6c2eaa862672', 'request_id': 'req-9003ce94-7d24-4f5f-81dd-b2a943575a21'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:40.952232 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-9003ce94-7d24-4f5f-81dd-b2a943575a21 admin admin] The token cache is being invalidate because project b231665ac65c4bf0b38f6c2eaa862672 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:03:40.952612 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-9003ce94-7d24-4f5f-81dd-b2a943575a21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:40.955677 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-9003ce94-7d24-4f5f-81dd-b2a943575a21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:40.962606 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 937/1859] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:03:40 2022] DELETE /identity/v3/projects/b231665ac65c4bf0b38f6c2eaa862672 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:40.964070 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9003ce94-7d24-4f5f-81dd-b2a943575a21 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:40.965849 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-9003ce94-7d24-4f5f-81dd-b2a943575a21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:41.004559 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e92b2a-a6ac-4707-93a9-08b958f3345d admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:41.011681 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dda0fb-e1a3-465a-8fbc-8467e1ae0613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:41.012652 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dda0fb-e1a3-465a-8fbc-8467e1ae0613 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:41.016074 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dda0fb-e1a3-465a-8fbc-8467e1ae0613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:41.016633 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dda0fb-e1a3-465a-8fbc-8467e1ae0613 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:41.017736 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dda0fb-e1a3-465a-8fbc-8467e1ae0613 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:41.019865 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dda0fb-e1a3-465a-8fbc-8467e1ae0613 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:41.034653 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 923/1860] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:40 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 22:03:41.038533 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dda0fb-e1a3-465a-8fbc-8467e1ae0613 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:41.040497 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-c0dda0fb-e1a3-465a-8fbc-8467e1ae0613 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:41.055790 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9003ce94-7d24-4f5f-81dd-b2a943575a21 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:41.057841 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dda0fb-e1a3-465a-8fbc-8467e1ae0613 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:41.063152 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6dab66-d9fd-4a51-900c-b13f38238668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:41.063696 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd7401-cd22-4b87-a856-dda76f95e178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:41.064430 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6dab66-d9fd-4a51-900c-b13f38238668 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:41.064742 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6dab66-d9fd-4a51-900c-b13f38238668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:41.065118 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6dab66-d9fd-4a51-900c-b13f38238668 admin admin] PATH_INFO: `/v3/projects/0fd0d297527b4701bb31615d46055539/users/18b1f026a28e42dd81a741d024f1d4fb/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:41.065805 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd7401-cd22-4b87-a856-dda76f95e178 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:41.066235 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd7401-cd22-4b87-a856-dda76f95e178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:41.066657 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd7401-cd22-4b87-a856-dda76f95e178 admin admin] PATH_INFO: `/v3/users/d39df364b9c245e8bfc55d4e66414aa6` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:41.068646 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fd7401-cd22-4b87-a856-dda76f95e178 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d39df364b9c245e8bfc55d4e66414aa6)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:41.070704 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fd7401-cd22-4b87-a856-dda76f95e178 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:41.080276 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6dab66-d9fd-4a51-900c-b13f38238668 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fd0d297527b4701bb31615d46055539, user_id=18b1f026a28e42dd81a741d024f1d4fb, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:41.084900 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6dab66-d9fd-4a51-900c-b13f38238668 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:41.103633 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 924/1861] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:03:41 2022] PUT /identity/v3/projects/0fd0d297527b4701bb31615d46055539/users/18b1f026a28e42dd81a741d024f1d4fb/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:03:41.142783 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-f1fd7401-cd22-4b87-a856-dda76f95e178 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd39df364b9c245e8bfc55d4e66414aa6'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:41.151823 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-f1fd7401-cd22-4b87-a856-dda76f95e178 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd39df364b9c245e8bfc55d4e66414aa6'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:41.170653 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-f1fd7401-cd22-4b87-a856-dda76f95e178 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd39df364b9c245e8bfc55d4e66414aa6'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:41.171754 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-f1fd7401-cd22-4b87-a856-dda76f95e178 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd39df364b9c245e8bfc55d4e66414aa6'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:41.172480 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-f1fd7401-cd22-4b87-a856-dda76f95e178 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd39df364b9c245e8bfc55d4e66414aa6'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:41.194451 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 938/1862] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:03:40 2022] DELETE /identity/v3/users/d39df364b9c245e8bfc55d4e66414aa6 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:41.657300 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6dab66-d9fd-4a51-900c-b13f38238668 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:41.660236 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-7d6dab66-d9fd-4a51-900c-b13f38238668 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:41.797922 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6dab66-d9fd-4a51-900c-b13f38238668 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:41.876047 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15302e6b-abaf-4eb4-a24e-558316ef1fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:41.876983 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15302e6b-abaf-4eb4-a24e-558316ef1fdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:41.877276 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15302e6b-abaf-4eb4-a24e-558316ef1fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:41.877554 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15302e6b-abaf-4eb4-a24e-558316ef1fdb admin admin] PATH_INFO: `/v3/projects/545ce39f2a324c4786ffc887f1a1b630` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:41.878786 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15302e6b-abaf-4eb4-a24e-558316ef1fdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=545ce39f2a324c4786ffc887f1a1b630)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:41.880756 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15302e6b-abaf-4eb4-a24e-558316ef1fdb admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:41.893438 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:41.932175 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-15302e6b-abaf-4eb4-a24e-558316ef1fdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '545ce39f2a324c4786ffc887f1a1b630', 'request_id': 'req-15302e6b-abaf-4eb4-a24e-558316ef1fdb'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:41.940949 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-15302e6b-abaf-4eb4-a24e-558316ef1fdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '545ce39f2a324c4786ffc887f1a1b630', 'request_id': 'req-15302e6b-abaf-4eb4-a24e-558316ef1fdb'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:41.948325 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-15302e6b-abaf-4eb4-a24e-558316ef1fdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '545ce39f2a324c4786ffc887f1a1b630', 'request_id': 'req-15302e6b-abaf-4eb4-a24e-558316ef1fdb'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:41.960682 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-15302e6b-abaf-4eb4-a24e-558316ef1fdb admin admin] The token cache is being invalidate because project 545ce39f2a324c4786ffc887f1a1b630 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:03:41.960984 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-15302e6b-abaf-4eb4-a24e-558316ef1fdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:41.961746 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-15302e6b-abaf-4eb4-a24e-558316ef1fdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:41.963168 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 925/1863] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:03:41 2022] DELETE /identity/v3/projects/545ce39f2a324c4786ffc887f1a1b630 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:41.968499 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75114d-8e82-411c-9554-3f018970f567 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:41.968996 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75114d-8e82-411c-9554-3f018970f567 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:41.969128 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75114d-8e82-411c-9554-3f018970f567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:41.992861 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-cc75114d-8e82-411c-9554-3f018970f567 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:41.994289 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-cc75114d-8e82-411c-9554-3f018970f567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:42.049552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:42.085013 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5558bbc7-be25-4d9c-adae-8310a98bd228 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:42.085331 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5558bbc7-be25-4d9c-adae-8310a98bd228 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:42.085612 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5558bbc7-be25-4d9c-adae-8310a98bd228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:42.086260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 939/1864] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:42.101662 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c9160-0a64-4ade-ad17-99cfce5e7126 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:42.102061 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c9160-0a64-4ade-ad17-99cfce5e7126 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:42.102617 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c9160-0a64-4ade-ad17-99cfce5e7126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:42.132298 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-5558bbc7-be25-4d9c-adae-8310a98bd228 None None] MFA Rules not processed for user `18b1f026a28e42dd81a741d024f1d4fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:42.133571 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-5558bbc7-be25-4d9c-adae-8310a98bd228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:42.134157 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-4b0c9160-0a64-4ade-ad17-99cfce5e7126 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:42.136202 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-4b0c9160-0a64-4ade-ad17-99cfce5e7126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:42.154190 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 940/1865] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:42.163290 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0c9160-0a64-4ade-ad17-99cfce5e7126 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:42.181041 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0c9160-0a64-4ade-ad17-99cfce5e7126 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:42.191316 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e645169e-56da-4f19-a213-f1b0fe647a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:42.194678 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645169e-56da-4f19-a213-f1b0fe647a82 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:42.196369 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645169e-56da-4f19-a213-f1b0fe647a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:42.196935 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645169e-56da-4f19-a213-f1b0fe647a82 admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:42.198693 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e645169e-56da-4f19-a213-f1b0fe647a82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:03:42.199219 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e645169e-56da-4f19-a213-f1b0fe647a82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:42.201551 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e645169e-56da-4f19-a213-f1b0fe647a82 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:42.221045 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 941/1866] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:03:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:03:42.226444 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e645169e-56da-4f19-a213-f1b0fe647a82 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:42.232638 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 926/1867] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:42 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:03:42.244704 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5558bbc7-be25-4d9c-adae-8310a98bd228 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:42.247732 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e645169e-56da-4f19-a213-f1b0fe647a82 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:42.248722 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-5558bbc7-be25-4d9c-adae-8310a98bd228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:42.255164 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b65581-e391-4f49-a36e-85c847983860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:42.256117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b65581-e391-4f49-a36e-85c847983860 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:42.256117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b65581-e391-4f49-a36e-85c847983860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:42.256414 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b65581-e391-4f49-a36e-85c847983860 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:42.257095 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b65581-e391-4f49-a36e-85c847983860 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:42.259426 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b65581-e391-4f49-a36e-85c847983860 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:42.312119 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 942/1868] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:03:42 2022] POST /identity/v3/projects => generated 381 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:03:42.313494 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b65581-e391-4f49-a36e-85c847983860 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:42.345476 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5558bbc7-be25-4d9c-adae-8310a98bd228 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:42.392616 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:42.401413 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b65581-e391-4f49-a36e-85c847983860 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:42.406703 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcb4065-d98a-4320-82fa-fda278e0adbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:42.407220 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25793d1e-e94b-4124-bba5-09e3012513f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:42.407441 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb4065-d98a-4320-82fa-fda278e0adbc service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:42.407718 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb4065-d98a-4320-82fa-fda278e0adbc service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:42.408031 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb4065-d98a-4320-82fa-fda278e0adbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:42.408588 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25793d1e-e94b-4124-bba5-09e3012513f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:42.408885 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25793d1e-e94b-4124-bba5-09e3012513f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:42.409168 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25793d1e-e94b-4124-bba5-09e3012513f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:42.409933 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25793d1e-e94b-4124-bba5-09e3012513f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:42.410109 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-dbcb4065-d98a-4320-82fa-fda278e0adbc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:42.411907 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25793d1e-e94b-4124-bba5-09e3012513f2 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:42.441216 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 943/1869] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:03:42 2022] POST /identity/v3/users => generated 291 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:03:42.442213 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25793d1e-e94b-4124-bba5-09e3012513f2 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:42.454630 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25793d1e-e94b-4124-bba5-09e3012513f2 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:42.460422 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1109a70f-7b20-4835-928a-acc204d33567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:42.461000 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109a70f-7b20-4835-928a-acc204d33567 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:42.461287 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109a70f-7b20-4835-928a-acc204d33567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:42.461567 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109a70f-7b20-4835-928a-acc204d33567 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:42.462346 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1109a70f-7b20-4835-928a-acc204d33567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:42.464714 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1109a70f-7b20-4835-928a-acc204d33567 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:42.477000 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 944/1870] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 22:03:42.480094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcb4065-d98a-4320-82fa-fda278e0adbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:42.482289 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1109a70f-7b20-4835-928a-acc204d33567 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:42.483011 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcb4065-d98a-4320-82fa-fda278e0adbc service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:42.495082 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 927/1871] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:42 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 22:03:42.496843 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1109a70f-7b20-4835-928a-acc204d33567 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:42.501924 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f34a8ef-8b65-47e2-ac6e-fbd7b3fc8a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:42.502577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34a8ef-8b65-47e2-ac6e-fbd7b3fc8a3d admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:42.502866 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34a8ef-8b65-47e2-ac6e-fbd7b3fc8a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:42.503171 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34a8ef-8b65-47e2-ac6e-fbd7b3fc8a3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:42.503940 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f34a8ef-8b65-47e2-ac6e-fbd7b3fc8a3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:42.505528 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f34a8ef-8b65-47e2-ac6e-fbd7b3fc8a3d admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:42.516345 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 945/1872] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 22:03:42.520647 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcb4065-d98a-4320-82fa-fda278e0adbc service cinder] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:42.532050 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcb4065-d98a-4320-82fa-fda278e0adbc service cinder] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:42.537169 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9114aa00-1760-4540-8e48-0be5b22624ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:42.538092 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114aa00-1760-4540-8e48-0be5b22624ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:42.538389 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114aa00-1760-4540-8e48-0be5b22624ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:42.538791 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114aa00-1760-4540-8e48-0be5b22624ad admin admin] PATH_INFO: `/v3/projects/2d41f93c073342fd932a55ac822d5e42/users/e12398ee404d4581829beb42570c27ec/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:42.555693 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9114aa00-1760-4540-8e48-0be5b22624ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d41f93c073342fd932a55ac822d5e42, user_id=e12398ee404d4581829beb42570c27ec, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:42.558436 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9114aa00-1760-4540-8e48-0be5b22624ad admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:42.574339 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 928/1873] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:03:42 2022] PUT /identity/v3/projects/2d41f93c073342fd932a55ac822d5e42/users/e12398ee404d4581829beb42570c27ec/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:42.586034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1527275-8f58-4c83-b8d5-2d5324bf0cd8 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:42.586432 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1527275-8f58-4c83-b8d5-2d5324bf0cd8 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:42.586787 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1527275-8f58-4c83-b8d5-2d5324bf0cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:42.615028 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-f1527275-8f58-4c83-b8d5-2d5324bf0cd8 None None] MFA Rules not processed for user `e12398ee404d4581829beb42570c27ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:42.616448 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-f1527275-8f58-4c83-b8d5-2d5324bf0cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:42.688294 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:42.696497 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 946/1874] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:42.698224 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9114aa00-1760-4540-8e48-0be5b22624ad admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:42.787239 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9114aa00-1760-4540-8e48-0be5b22624ad admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:42.794162 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced48a71-d3f4-4201-a35e-13aa6fcb6552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:42.795752 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced48a71-d3f4-4201-a35e-13aa6fcb6552 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:42.796113 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced48a71-d3f4-4201-a35e-13aa6fcb6552 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:42.796467 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced48a71-d3f4-4201-a35e-13aa6fcb6552 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:42.800087 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced48a71-d3f4-4201-a35e-13aa6fcb6552 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:42.802151 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced48a71-d3f4-4201-a35e-13aa6fcb6552 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:42.842959 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 929/1875] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:03:42 2022] POST /identity/v3/projects => generated 383 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:03:42.846710 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1527275-8f58-4c83-b8d5-2d5324bf0cd8 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:42.859061 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1527275-8f58-4c83-b8d5-2d5324bf0cd8 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:42.917637 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a168652c-0d3f-44e3-a9a0-4ba745cec48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:42.918314 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a168652c-0d3f-44e3-a9a0-4ba745cec48a admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:42.918599 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a168652c-0d3f-44e3-a9a0-4ba745cec48a admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:42.918876 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a168652c-0d3f-44e3-a9a0-4ba745cec48a admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:42.919608 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a168652c-0d3f-44e3-a9a0-4ba745cec48a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:42.921524 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a168652c-0d3f-44e3-a9a0-4ba745cec48a admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:42.942147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 947/1876] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:03:42 2022] POST /identity/v3/users => generated 298 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:42.945326 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced48a71-d3f4-4201-a35e-13aa6fcb6552 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:42.956202 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced48a71-d3f4-4201-a35e-13aa6fcb6552 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:42.960757 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e578581-9530-4f7d-b98e-64f332e5ffca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:42.961413 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e578581-9530-4f7d-b98e-64f332e5ffca admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:42.961691 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e578581-9530-4f7d-b98e-64f332e5ffca admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:42.961965 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e578581-9530-4f7d-b98e-64f332e5ffca admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:42.962708 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e578581-9530-4f7d-b98e-64f332e5ffca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:42.964485 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e578581-9530-4f7d-b98e-64f332e5ffca admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:42.974830 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 930/1877] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:42.979101 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a168652c-0d3f-44e3-a9a0-4ba745cec48a admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:42.989013 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a168652c-0d3f-44e3-a9a0-4ba745cec48a admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:42.993730 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92086fc6-4cbf-497b-bf21-ff5535d30dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:42.995190 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92086fc6-4cbf-497b-bf21-ff5535d30dc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:42.995558 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92086fc6-4cbf-497b-bf21-ff5535d30dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:42.995933 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92086fc6-4cbf-497b-bf21-ff5535d30dc0 admin admin] PATH_INFO: `/v3/projects/c8064409f0ef463b8813d0cf744bb89c/users/3c29c605735c4530ae6df0b84f5f3dab/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:43.011361 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92086fc6-4cbf-497b-bf21-ff5535d30dc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8064409f0ef463b8813d0cf744bb89c, user_id=3c29c605735c4530ae6df0b84f5f3dab, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:43.014050 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92086fc6-4cbf-497b-bf21-ff5535d30dc0 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:43.027772 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 948/1878] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:03:42 2022] PUT /identity/v3/projects/c8064409f0ef463b8813d0cf744bb89c/users/3c29c605735c4530ae6df0b84f5f3dab/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:43.031670 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e578581-9530-4f7d-b98e-64f332e5ffca admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:43.041468 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e578581-9530-4f7d-b98e-64f332e5ffca admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:43.085434 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:43.100039 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3726acb1-980a-4aaf-92fb-eb613d05ec5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:43.101278 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3726acb1-980a-4aaf-92fb-eb613d05ec5e admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:43.101761 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3726acb1-980a-4aaf-92fb-eb613d05ec5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:43.103364 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3726acb1-980a-4aaf-92fb-eb613d05ec5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:43.103364 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3726acb1-980a-4aaf-92fb-eb613d05ec5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:43.104895 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3726acb1-980a-4aaf-92fb-eb613d05ec5e admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:43.115043 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 931/1879] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:43 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:43.119408 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92086fc6-4cbf-497b-bf21-ff5535d30dc0 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:43.130629 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92086fc6-4cbf-497b-bf21-ff5535d30dc0 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:43.135980 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f14034b-83d4-4022-9cde-b8756e4784b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:43.137290 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14034b-83d4-4022-9cde-b8756e4784b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:43.137644 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14034b-83d4-4022-9cde-b8756e4784b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:43.137987 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14034b-83d4-4022-9cde-b8756e4784b0 admin admin] PATH_INFO: `/v3/projects/c8064409f0ef463b8813d0cf744bb89c/users/3c29c605735c4530ae6df0b84f5f3dab/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:43.140172 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f14034b-83d4-4022-9cde-b8756e4784b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8064409f0ef463b8813d0cf744bb89c, user_id=3c29c605735c4530ae6df0b84f5f3dab, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:43.142859 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f14034b-83d4-4022-9cde-b8756e4784b0 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:43.154455 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 949/1880] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:03:43 2022] PUT /identity/v3/projects/c8064409f0ef463b8813d0cf744bb89c/users/3c29c605735c4530ae6df0b84f5f3dab/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:43.164648 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f8697-3a08-4347-8b75-1b3e32a15587 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:43.164947 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f8697-3a08-4347-8b75-1b3e32a15587 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:43.165234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f8697-3a08-4347-8b75-1b3e32a15587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:43.192534 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-f84f8697-3a08-4347-8b75-1b3e32a15587 None None] MFA Rules not processed for user `3c29c605735c4530ae6df0b84f5f3dab`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:43.193679 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-f84f8697-3a08-4347-8b75-1b3e32a15587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:43.262548 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 932/1881] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:43 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:43.274819 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f14034b-83d4-4022-9cde-b8756e4784b0 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:43.333478 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f14034b-83d4-4022-9cde-b8756e4784b0 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:43.337978 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cddc7e8-9574-4da3-9818-ba122b698089 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:43.339558 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cddc7e8-9574-4da3-9818-ba122b698089 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:43.339944 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cddc7e8-9574-4da3-9818-ba122b698089 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:43.341006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cddc7e8-9574-4da3-9818-ba122b698089 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:43.350475 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cddc7e8-9574-4da3-9818-ba122b698089 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:43.352255 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cddc7e8-9574-4da3-9818-ba122b698089 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:43.362759 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 950/1882] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:43 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:03:43.468559 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84f8697-3a08-4347-8b75-1b3e32a15587 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:43.471130 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-f84f8697-3a08-4347-8b75-1b3e32a15587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:43.494094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84f8697-3a08-4347-8b75-1b3e32a15587 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:43.498146 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f22ac9-6cec-43ba-8bd1-0f79b1e6be0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:43.498808 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f22ac9-6cec-43ba-8bd1-0f79b1e6be0e service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:43.499091 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f22ac9-6cec-43ba-8bd1-0f79b1e6be0e service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:43.499393 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f22ac9-6cec-43ba-8bd1-0f79b1e6be0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:43.563650 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f22ac9-6cec-43ba-8bd1-0f79b1e6be0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:43.565091 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f22ac9-6cec-43ba-8bd1-0f79b1e6be0e service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:43.574879 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 933/1883] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:43 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:03:43.969647 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cddc7e8-9574-4da3-9818-ba122b698089 service cinder] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:44.110141 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:44.126763 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cddc7e8-9574-4da3-9818-ba122b698089 service cinder] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:44.137310 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b263554-de81-488a-a652-86d0a1bde27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:44.138534 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b263554-de81-488a-a652-86d0a1bde27c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:44.138997 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b263554-de81-488a-a652-86d0a1bde27c admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:44.139497 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b263554-de81-488a-a652-86d0a1bde27c admin admin] PATH_INFO: `/v3/users/e12398ee404d4581829beb42570c27ec` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:44.141242 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b263554-de81-488a-a652-86d0a1bde27c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e12398ee404d4581829beb42570c27ec)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:44.143794 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b263554-de81-488a-a652-86d0a1bde27c admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:44.211611 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-0b263554-de81-488a-a652-86d0a1bde27c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e12398ee404d4581829beb42570c27ec'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:44.221676 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-0b263554-de81-488a-a652-86d0a1bde27c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e12398ee404d4581829beb42570c27ec'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:44.238375 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-0b263554-de81-488a-a652-86d0a1bde27c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e12398ee404d4581829beb42570c27ec'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:44.239789 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-0b263554-de81-488a-a652-86d0a1bde27c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e12398ee404d4581829beb42570c27ec'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:44.240473 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-0b263554-de81-488a-a652-86d0a1bde27c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e12398ee404d4581829beb42570c27ec'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:44.255893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 951/1884] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:03:43 2022] DELETE /identity/v3/users/e12398ee404d4581829beb42570c27ec => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:44.260687 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71838c-454e-44b6-8e45-80f284c403c1 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:44.261208 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71838c-454e-44b6-8e45-80f284c403c1 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:44.261603 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71838c-454e-44b6-8e45-80f284c403c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:44.280979 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-2c71838c-454e-44b6-8e45-80f284c403c1 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:44.282577 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-2c71838c-454e-44b6-8e45-80f284c403c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:44.363359 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 934/1885] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:44.403683 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b263554-de81-488a-a652-86d0a1bde27c admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:44.405648 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-0b263554-de81-488a-a652-86d0a1bde27c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:44.495626 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b263554-de81-488a-a652-86d0a1bde27c admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:44.512513 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72966d46-3deb-4754-9f08-b452e04a510a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:44.513197 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72966d46-3deb-4754-9f08-b452e04a510a service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:44.513483 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72966d46-3deb-4754-9f08-b452e04a510a service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:44.513763 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72966d46-3deb-4754-9f08-b452e04a510a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:44.535098 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72966d46-3deb-4754-9f08-b452e04a510a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:44.536691 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72966d46-3deb-4754-9f08-b452e04a510a service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:44.564670 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 952/1886] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:03:45.165147 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c71838c-454e-44b6-8e45-80f284c403c1 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:45.167119 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-2c71838c-454e-44b6-8e45-80f284c403c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:45.182205 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c71838c-454e-44b6-8e45-80f284c403c1 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:45.188473 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd1b3f-60be-43ca-8b6b-0495cc2cc9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:45.189504 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd1b3f-60be-43ca-8b6b-0495cc2cc9a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:45.189803 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd1b3f-60be-43ca-8b6b-0495cc2cc9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:45.190143 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd1b3f-60be-43ca-8b6b-0495cc2cc9a2 admin admin] PATH_INFO: `/v3/projects/2d41f93c073342fd932a55ac822d5e42` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:45.191521 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fd1b3f-60be-43ca-8b6b-0495cc2cc9a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d41f93c073342fd932a55ac822d5e42)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:45.193499 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fd1b3f-60be-43ca-8b6b-0495cc2cc9a2 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:45.195289 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:45.261177 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-75fd1b3f-60be-43ca-8b6b-0495cc2cc9a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d41f93c073342fd932a55ac822d5e42', 'request_id': 'req-75fd1b3f-60be-43ca-8b6b-0495cc2cc9a2'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:45.277963 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-75fd1b3f-60be-43ca-8b6b-0495cc2cc9a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d41f93c073342fd932a55ac822d5e42', 'request_id': 'req-75fd1b3f-60be-43ca-8b6b-0495cc2cc9a2'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:45.288811 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-75fd1b3f-60be-43ca-8b6b-0495cc2cc9a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d41f93c073342fd932a55ac822d5e42', 'request_id': 'req-75fd1b3f-60be-43ca-8b6b-0495cc2cc9a2'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:45.304411 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-75fd1b3f-60be-43ca-8b6b-0495cc2cc9a2 admin admin] The token cache is being invalidate because project 2d41f93c073342fd932a55ac822d5e42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:03:45.304710 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-75fd1b3f-60be-43ca-8b6b-0495cc2cc9a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:45.305390 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-75fd1b3f-60be-43ca-8b6b-0495cc2cc9a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:45.307209 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 935/1887] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:03:45 2022] DELETE /identity/v3/projects/2d41f93c073342fd932a55ac822d5e42 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:45.311064 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72966d46-3deb-4754-9f08-b452e04a510a service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:45.321199 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72966d46-3deb-4754-9f08-b452e04a510a service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:45.413101 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5985ff0a-54f1-4105-9f93-eed6c00c31cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:45.414137 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5985ff0a-54f1-4105-9f93-eed6c00c31cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:45.414421 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5985ff0a-54f1-4105-9f93-eed6c00c31cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:45.414697 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5985ff0a-54f1-4105-9f93-eed6c00c31cb admin admin] PATH_INFO: `/v3/users/3c29c605735c4530ae6df0b84f5f3dab` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:45.418797 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5985ff0a-54f1-4105-9f93-eed6c00c31cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c29c605735c4530ae6df0b84f5f3dab)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:45.420842 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5985ff0a-54f1-4105-9f93-eed6c00c31cb admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:45.446023 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:45.503636 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-5985ff0a-54f1-4105-9f93-eed6c00c31cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c29c605735c4530ae6df0b84f5f3dab'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:45.513965 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-5985ff0a-54f1-4105-9f93-eed6c00c31cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c29c605735c4530ae6df0b84f5f3dab'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:45.559400 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-5985ff0a-54f1-4105-9f93-eed6c00c31cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c29c605735c4530ae6df0b84f5f3dab'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:45.560555 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-5985ff0a-54f1-4105-9f93-eed6c00c31cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c29c605735c4530ae6df0b84f5f3dab'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:45.561512 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-5985ff0a-54f1-4105-9f93-eed6c00c31cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c29c605735c4530ae6df0b84f5f3dab'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:45.585651 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 953/1888] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:03:45 2022] DELETE /identity/v3/users/3c29c605735c4530ae6df0b84f5f3dab => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:46.194067 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd1b3f-60be-43ca-8b6b-0495cc2cc9a2 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:46.196471 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-75fd1b3f-60be-43ca-8b6b-0495cc2cc9a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:46.352677 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd1b3f-60be-43ca-8b6b-0495cc2cc9a2 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:46.449542 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5985ff0a-54f1-4105-9f93-eed6c00c31cb admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:46.451865 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-5985ff0a-54f1-4105-9f93-eed6c00c31cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:46.475219 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc6892-6895-4941-9d8c-e063175d7a8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:46.476313 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc6892-6895-4941-9d8c-e063175d7a8c service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:46.476589 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc6892-6895-4941-9d8c-e063175d7a8c service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:46.476973 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc6892-6895-4941-9d8c-e063175d7a8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:46.478976 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-10bc6892-6895-4941-9d8c-e063175d7a8c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:46.546972 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5985ff0a-54f1-4105-9f93-eed6c00c31cb admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:46.553381 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd10687-6979-4716-bdcb-eba56bc9f77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:46.554275 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd10687-6979-4716-bdcb-eba56bc9f77f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:46.554559 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd10687-6979-4716-bdcb-eba56bc9f77f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:46.554835 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd10687-6979-4716-bdcb-eba56bc9f77f admin admin] PATH_INFO: `/v3/projects/c8064409f0ef463b8813d0cf744bb89c` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:46.556146 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd10687-6979-4716-bdcb-eba56bc9f77f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8064409f0ef463b8813d0cf744bb89c)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:46.559135 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd10687-6979-4716-bdcb-eba56bc9f77f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:46.583901 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:46.586760 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bc6892-6895-4941-9d8c-e063175d7a8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:46.588753 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bc6892-6895-4941-9d8c-e063175d7a8c service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:46.611762 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 936/1889] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:46 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:03:46.613557 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e83e57-4de4-4be7-9616-f1579af45254 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:46.614042 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e83e57-4de4-4be7-9616-f1579af45254 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:46.614526 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e83e57-4de4-4be7-9616-f1579af45254 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:46.616800 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 937/1890] 10.222.0.13 () {54 vars in 921 bytes} [Tue Jun 7 22:03:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 22:03:46.636103 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-6dd10687-6979-4716-bdcb-eba56bc9f77f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8064409f0ef463b8813d0cf744bb89c', 'request_id': 'req-6dd10687-6979-4716-bdcb-eba56bc9f77f'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:46.648638 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c984bd-9d6f-4e2e-9d36-747c31f60580 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:46.648995 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c984bd-9d6f-4e2e-9d36-747c31f60580 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:46.649562 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c984bd-9d6f-4e2e-9d36-747c31f60580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:46.668798 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-6dd10687-6979-4716-bdcb-eba56bc9f77f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8064409f0ef463b8813d0cf744bb89c', 'request_id': 'req-6dd10687-6979-4716-bdcb-eba56bc9f77f'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:46.680294 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-6dd10687-6979-4716-bdcb-eba56bc9f77f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8064409f0ef463b8813d0cf744bb89c', 'request_id': 'req-6dd10687-6979-4716-bdcb-eba56bc9f77f'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:46.688074 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-31c984bd-9d6f-4e2e-9d36-747c31f60580 None None] MFA Rules not processed for user `dbe0303ea0764347bfb054870ebba41f`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:46.695683 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-31c984bd-9d6f-4e2e-9d36-747c31f60580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:46.707258 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-6dd10687-6979-4716-bdcb-eba56bc9f77f admin admin] The token cache is being invalidate because project c8064409f0ef463b8813d0cf744bb89c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:03:46.709033 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-6dd10687-6979-4716-bdcb-eba56bc9f77f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:46.711419 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-6dd10687-6979-4716-bdcb-eba56bc9f77f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:46.716839 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 954/1891] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:03:46 2022] DELETE /identity/v3/projects/c8064409f0ef463b8813d0cf744bb89c => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:46.721479 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49780d86-c934-4a8a-950a-aa3fe41ae05c None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:46.721839 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49780d86-c934-4a8a-950a-aa3fe41ae05c None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:46.722216 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49780d86-c934-4a8a-950a-aa3fe41ae05c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:46.784629 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-49780d86-c934-4a8a-950a-aa3fe41ae05c None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:46.786770 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-49780d86-c934-4a8a-950a-aa3fe41ae05c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:46.840908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:46.890496 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 938/1892] 10.222.0.13 () {58 vars in 1018 bytes} [Tue Jun 7 22:03:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:03:46.929466 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c984bd-9d6f-4e2e-9d36-747c31f60580 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:46.931399 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-31c984bd-9d6f-4e2e-9d36-747c31f60580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:46.957569 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c984bd-9d6f-4e2e-9d36-747c31f60580 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:46.990613 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbc3363-8c3e-4e2f-aba1-c5eb0c2ebf28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:46.990613 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbc3363-8c3e-4e2f-aba1-c5eb0c2ebf28 service nova] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:46.990613 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbc3363-8c3e-4e2f-aba1-c5eb0c2ebf28 service nova] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:46.990613 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbc3363-8c3e-4e2f-aba1-c5eb0c2ebf28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:46.992376 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 955/1893] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:46.993910 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c6192-6a03-493f-b5a9-40d7ae24aa81 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:46.994307 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c6192-6a03-493f-b5a9-40d7ae24aa81 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:46.995211 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c6192-6a03-493f-b5a9-40d7ae24aa81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:47.027362 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbc3363-8c3e-4e2f-aba1-c5eb0c2ebf28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:47.029023 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbc3363-8c3e-4e2f-aba1-c5eb0c2ebf28 service nova] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:47.044930 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-b66c6192-6a03-493f-b5a9-40d7ae24aa81 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:47.046160 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-b66c6192-6a03-493f-b5a9-40d7ae24aa81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:47.050652 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 939/1894] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:03:47.070198 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 956/1895] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:47.074225 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbc3363-8c3e-4e2f-aba1-c5eb0c2ebf28 service nova] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:47.090690 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbc3363-8c3e-4e2f-aba1-c5eb0c2ebf28 service nova] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:47.098408 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde5cd7e-818c-42e1-aae7-a7a116397855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:47.099312 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde5cd7e-818c-42e1-aae7-a7a116397855 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:47.099602 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde5cd7e-818c-42e1-aae7-a7a116397855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:47.099845 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde5cd7e-818c-42e1-aae7-a7a116397855 admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:47.100584 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde5cd7e-818c-42e1-aae7-a7a116397855 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:03:47.102981 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde5cd7e-818c-42e1-aae7-a7a116397855 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:47.109254 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde5cd7e-818c-42e1-aae7-a7a116397855 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:47.130965 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 940/1896] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:03:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:03:47.131988 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c6192-6a03-493f-b5a9-40d7ae24aa81 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:47.142332 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c6192-6a03-493f-b5a9-40d7ae24aa81 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:47.149772 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5325d651-db1d-4be0-a12c-1ec4979e3d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:47.150803 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325d651-db1d-4be0-a12c-1ec4979e3d98 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:47.150979 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325d651-db1d-4be0-a12c-1ec4979e3d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:47.151113 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325d651-db1d-4be0-a12c-1ec4979e3d98 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:47.151877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5325d651-db1d-4be0-a12c-1ec4979e3d98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:47.153788 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5325d651-db1d-4be0-a12c-1ec4979e3d98 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:47.190614 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 957/1897] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:03:47 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:03:47.194805 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde5cd7e-818c-42e1-aae7-a7a116397855 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:47.207649 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde5cd7e-818c-42e1-aae7-a7a116397855 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:47.275054 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d40309-91a9-47ed-b678-b4a7c8e602a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:47.275847 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d40309-91a9-47ed-b678-b4a7c8e602a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:47.276140 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d40309-91a9-47ed-b678-b4a7c8e602a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:47.276501 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d40309-91a9-47ed-b678-b4a7c8e602a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:47.277417 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d40309-91a9-47ed-b678-b4a7c8e602a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:47.279442 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d40309-91a9-47ed-b678-b4a7c8e602a0 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:47.323364 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 941/1898] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:03:47 2022] POST /identity/v3/users => generated 292 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:47.324802 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5325d651-db1d-4be0-a12c-1ec4979e3d98 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:47.338394 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5325d651-db1d-4be0-a12c-1ec4979e3d98 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:47.355529 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c921c-249d-473e-8b17-f03366809b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:47.356229 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c921c-249d-473e-8b17-f03366809b20 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:47.356517 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c921c-249d-473e-8b17-f03366809b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:47.356794 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c921c-249d-473e-8b17-f03366809b20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:47.357562 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711c921c-249d-473e-8b17-f03366809b20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:47.359253 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711c921c-249d-473e-8b17-f03366809b20 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:47.371074 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 958/1899] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:47.375160 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d40309-91a9-47ed-b678-b4a7c8e602a0 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:47.385234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d40309-91a9-47ed-b678-b4a7c8e602a0 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:47.389943 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ad41a-472b-4746-9efc-49a6be4469b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:47.390495 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ad41a-472b-4746-9efc-49a6be4469b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:47.390775 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ad41a-472b-4746-9efc-49a6be4469b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:47.391146 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ad41a-472b-4746-9efc-49a6be4469b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:47.393357 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058ad41a-472b-4746-9efc-49a6be4469b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:47.394987 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058ad41a-472b-4746-9efc-49a6be4469b7 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:47.408234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 942/1900] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:47.411297 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c921c-249d-473e-8b17-f03366809b20 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:47.425062 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c921c-249d-473e-8b17-f03366809b20 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:47.430049 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85abe84-2fe8-4aea-b9ef-edf6b2ab0cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:47.431142 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85abe84-2fe8-4aea-b9ef-edf6b2ab0cd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:47.431460 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85abe84-2fe8-4aea-b9ef-edf6b2ab0cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:47.431745 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85abe84-2fe8-4aea-b9ef-edf6b2ab0cd0 admin admin] PATH_INFO: `/v3/projects/4632120905db4c458b81720442355550/users/d165187193a94507bdaebfa52192dd52/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:47.448788 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85abe84-2fe8-4aea-b9ef-edf6b2ab0cd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4632120905db4c458b81720442355550, user_id=d165187193a94507bdaebfa52192dd52, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:47.451580 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85abe84-2fe8-4aea-b9ef-edf6b2ab0cd0 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:47.465594 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 959/1901] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:03:47 2022] PUT /identity/v3/projects/4632120905db4c458b81720442355550/users/d165187193a94507bdaebfa52192dd52/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:47.476658 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2694c-2c99-4e39-856d-cf335c4a5c61 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:47.476947 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2694c-2c99-4e39-856d-cf335c4a5c61 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:47.477224 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2694c-2c99-4e39-856d-cf335c4a5c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:47.504525 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-35a2694c-2c99-4e39-856d-cf335c4a5c61 None None] MFA Rules not processed for user `d165187193a94507bdaebfa52192dd52`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:47.505746 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-35a2694c-2c99-4e39-856d-cf335c4a5c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:47.534731 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:47.571735 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 943/1902] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:47.582604 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85abe84-2fe8-4aea-b9ef-edf6b2ab0cd0 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:47.584311 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a85abe84-2fe8-4aea-b9ef-edf6b2ab0cd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:47.642656 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85abe84-2fe8-4aea-b9ef-edf6b2ab0cd0 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:47.646321 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ef05c-99bb-4710-880b-22e925704165 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:47.646968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ef05c-99bb-4710-880b-22e925704165 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:47.647253 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ef05c-99bb-4710-880b-22e925704165 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:47.647568 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ef05c-99bb-4710-880b-22e925704165 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:47.656358 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369ef05c-99bb-4710-880b-22e925704165 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:47.657782 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369ef05c-99bb-4710-880b-22e925704165 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:47.667107 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 960/1903] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:03:47.686513 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a2694c-2c99-4e39-856d-cf335c4a5c61 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:47.747778 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a2694c-2c99-4e39-856d-cf335c4a5c61 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:47.752411 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02d165-a890-4012-be43-8cb8c0cc57d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:47.753309 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02d165-a890-4012-be43-8cb8c0cc57d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:47.753596 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02d165-a890-4012-be43-8cb8c0cc57d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:47.753873 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02d165-a890-4012-be43-8cb8c0cc57d4 admin admin] PATH_INFO: `/v3/users/d165187193a94507bdaebfa52192dd52` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:47.755078 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe02d165-a890-4012-be43-8cb8c0cc57d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d165187193a94507bdaebfa52192dd52)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:47.757028 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe02d165-a890-4012-be43-8cb8c0cc57d4 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:47.807069 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-fe02d165-a890-4012-be43-8cb8c0cc57d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd165187193a94507bdaebfa52192dd52'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:47.807793 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-fe02d165-a890-4012-be43-8cb8c0cc57d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd165187193a94507bdaebfa52192dd52'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:47.815565 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-fe02d165-a890-4012-be43-8cb8c0cc57d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd165187193a94507bdaebfa52192dd52'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:47.828586 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-fe02d165-a890-4012-be43-8cb8c0cc57d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd165187193a94507bdaebfa52192dd52'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:47.835572 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-fe02d165-a890-4012-be43-8cb8c0cc57d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd165187193a94507bdaebfa52192dd52'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:47.844277 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 944/1904] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:03:47 2022] DELETE /identity/v3/users/d165187193a94507bdaebfa52192dd52 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:47.848673 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac79901e-9788-4774-bedc-de2cdcab0019 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:47.848968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac79901e-9788-4774-bedc-de2cdcab0019 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:47.849270 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac79901e-9788-4774-bedc-de2cdcab0019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:47.869345 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-ac79901e-9788-4774-bedc-de2cdcab0019 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:47.870531 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-ac79901e-9788-4774-bedc-de2cdcab0019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:47.934525 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 961/1905] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:47.944125 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02d165-a890-4012-be43-8cb8c0cc57d4 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:47.945893 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-fe02d165-a890-4012-be43-8cb8c0cc57d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:48.015127 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02d165-a890-4012-be43-8cb8c0cc57d4 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:48.019521 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:48.020074 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b0a9f6-e257-44cf-8082-d8a1327a29bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:48.021465 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0a9f6-e257-44cf-8082-d8a1327a29bc service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:48.022019 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0a9f6-e257-44cf-8082-d8a1327a29bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:48.022490 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0a9f6-e257-44cf-8082-d8a1327a29bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:48.037592 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b0a9f6-e257-44cf-8082-d8a1327a29bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:48.048505 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b0a9f6-e257-44cf-8082-d8a1327a29bc service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:48.051264 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 945/1906] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 22:03:48.603045 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac79901e-9788-4774-bedc-de2cdcab0019 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:48.605084 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-ac79901e-9788-4774-bedc-de2cdcab0019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:48.617623 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac79901e-9788-4774-bedc-de2cdcab0019 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:48.624674 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a18cdd-9e12-4fc9-82a7-cd2745914c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:48.625562 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a18cdd-9e12-4fc9-82a7-cd2745914c96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:48.625849 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a18cdd-9e12-4fc9-82a7-cd2745914c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:48.626123 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a18cdd-9e12-4fc9-82a7-cd2745914c96 admin admin] PATH_INFO: `/v3/projects/4632120905db4c458b81720442355550` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:48.627355 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a18cdd-9e12-4fc9-82a7-cd2745914c96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4632120905db4c458b81720442355550)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:48.629443 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a18cdd-9e12-4fc9-82a7-cd2745914c96 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:48.631152 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:48.688865 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-32a18cdd-9e12-4fc9-82a7-cd2745914c96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4632120905db4c458b81720442355550', 'request_id': 'req-32a18cdd-9e12-4fc9-82a7-cd2745914c96'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:48.697351 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-32a18cdd-9e12-4fc9-82a7-cd2745914c96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4632120905db4c458b81720442355550', 'request_id': 'req-32a18cdd-9e12-4fc9-82a7-cd2745914c96'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:48.703107 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-32a18cdd-9e12-4fc9-82a7-cd2745914c96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4632120905db4c458b81720442355550', 'request_id': 'req-32a18cdd-9e12-4fc9-82a7-cd2745914c96'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:48.713860 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-32a18cdd-9e12-4fc9-82a7-cd2745914c96 admin admin] The token cache is being invalidate because project 4632120905db4c458b81720442355550 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:03:48.714151 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-32a18cdd-9e12-4fc9-82a7-cd2745914c96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:48.714775 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-32a18cdd-9e12-4fc9-82a7-cd2745914c96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:03:48.716343 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 962/1907] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:03:48 2022] DELETE /identity/v3/projects/4632120905db4c458b81720442355550 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:48.721164 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a4c583-7606-4614-95f5-a6196f6878bc None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:48.721657 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a4c583-7606-4614-95f5-a6196f6878bc None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:48.721744 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a4c583-7606-4614-95f5-a6196f6878bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:48.740570 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-24a4c583-7606-4614-95f5-a6196f6878bc None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:48.741775 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-24a4c583-7606-4614-95f5-a6196f6878bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:48.812646 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 946/1908] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:48.820319 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c1f1f-dff0-4a3e-a809-08ed8411200d None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:48.820608 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c1f1f-dff0-4a3e-a809-08ed8411200d None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:48.820884 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c1f1f-dff0-4a3e-a809-08ed8411200d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:48.838975 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-4b4c1f1f-dff0-4a3e-a809-08ed8411200d None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:48.840356 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-4b4c1f1f-dff0-4a3e-a809-08ed8411200d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:48.848091 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 963/1909] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:48.852328 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a4c583-7606-4614-95f5-a6196f6878bc None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:48.862504 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a4c583-7606-4614-95f5-a6196f6878bc None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:48.867072 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff3116c-0ce1-44e7-875c-8df4451e9c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:48.867758 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3116c-0ce1-44e7-875c-8df4451e9c06 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:48.868040 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3116c-0ce1-44e7-875c-8df4451e9c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:48.868313 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3116c-0ce1-44e7-875c-8df4451e9c06 admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:48.869035 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff3116c-0ce1-44e7-875c-8df4451e9c06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:03:48.870000 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff3116c-0ce1-44e7-875c-8df4451e9c06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:48.871899 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff3116c-0ce1-44e7-875c-8df4451e9c06 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:48.883255 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 947/1910] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:03:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:03:48.887595 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4c1f1f-dff0-4a3e-a809-08ed8411200d None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:48.896534 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4c1f1f-dff0-4a3e-a809-08ed8411200d None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:48.901749 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009d061-9c65-4f83-925b-1a6f074559e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:48.902387 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009d061-9c65-4f83-925b-1a6f074559e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:48.902668 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009d061-9c65-4f83-925b-1a6f074559e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:48.902943 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009d061-9c65-4f83-925b-1a6f074559e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:48.903672 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6009d061-9c65-4f83-925b-1a6f074559e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:48.905545 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6009d061-9c65-4f83-925b-1a6f074559e0 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:48.930336 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 964/1911] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:03:48 2022] POST /identity/v3/projects => generated 383 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:48.934555 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff3116c-0ce1-44e7-875c-8df4451e9c06 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:48.944412 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff3116c-0ce1-44e7-875c-8df4451e9c06 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:49.004343 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a540119-770e-449f-9346-abfdad833dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:49.005038 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a540119-770e-449f-9346-abfdad833dee admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:49.005354 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a540119-770e-449f-9346-abfdad833dee admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:49.005656 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a540119-770e-449f-9346-abfdad833dee admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:49.006369 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a540119-770e-449f-9346-abfdad833dee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:49.008452 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a540119-770e-449f-9346-abfdad833dee admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:49.031310 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 948/1912] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:03:48 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:49.035671 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009d061-9c65-4f83-925b-1a6f074559e0 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:49.047223 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009d061-9c65-4f83-925b-1a6f074559e0 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:49.052380 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94924f75-7651-4c37-95ac-249ccbe367f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:49.053185 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94924f75-7651-4c37-95ac-249ccbe367f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:49.053482 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94924f75-7651-4c37-95ac-249ccbe367f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:49.053761 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94924f75-7651-4c37-95ac-249ccbe367f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:49.054705 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94924f75-7651-4c37-95ac-249ccbe367f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:49.056287 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94924f75-7651-4c37-95ac-249ccbe367f4 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:49.071748 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 965/1913] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:49.072649 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a540119-770e-449f-9346-abfdad833dee admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:49.083564 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a540119-770e-449f-9346-abfdad833dee admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:49.089759 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da6383b-d587-4bc4-ae8f-982a92842a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:49.090431 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da6383b-d587-4bc4-ae8f-982a92842a1b admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:49.090728 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da6383b-d587-4bc4-ae8f-982a92842a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:49.090999 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da6383b-d587-4bc4-ae8f-982a92842a1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:49.091780 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da6383b-d587-4bc4-ae8f-982a92842a1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:49.093607 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da6383b-d587-4bc4-ae8f-982a92842a1b admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:49.105838 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 949/1914] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:49.110269 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94924f75-7651-4c37-95ac-249ccbe367f4 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:49.121460 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94924f75-7651-4c37-95ac-249ccbe367f4 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:49.127039 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931b893c-a512-4147-8512-ddfc0d8266d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:49.127869 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931b893c-a512-4147-8512-ddfc0d8266d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:49.128221 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931b893c-a512-4147-8512-ddfc0d8266d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:49.128556 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931b893c-a512-4147-8512-ddfc0d8266d0 admin admin] PATH_INFO: `/v3/projects/efaa643308264befaec40cb9ba5d4089/users/2d3c69e4fd6148f295369531caf98b65/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:49.147358 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931b893c-a512-4147-8512-ddfc0d8266d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efaa643308264befaec40cb9ba5d4089, user_id=2d3c69e4fd6148f295369531caf98b65, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:49.150003 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931b893c-a512-4147-8512-ddfc0d8266d0 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:49.167087 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 966/1915] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:03:49 2022] PUT /identity/v3/projects/efaa643308264befaec40cb9ba5d4089/users/2d3c69e4fd6148f295369531caf98b65/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:03:49.177346 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd0c949-213a-4340-96a7-557cffc34856 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:49.177673 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd0c949-213a-4340-96a7-557cffc34856 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:49.177974 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd0c949-213a-4340-96a7-557cffc34856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:49.206237 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-7bd0c949-213a-4340-96a7-557cffc34856 None None] MFA Rules not processed for user `2d3c69e4fd6148f295369531caf98b65`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:49.207535 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-7bd0c949-213a-4340-96a7-557cffc34856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:49.253494 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:49.274762 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 950/1916] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:03:49.279263 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931b893c-a512-4147-8512-ddfc0d8266d0 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:49.340363 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931b893c-a512-4147-8512-ddfc0d8266d0 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:49.344954 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded5510a-1081-4782-856f-d7e6d8b8b927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:49.345600 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5510a-1081-4782-856f-d7e6d8b8b927 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:49.345884 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5510a-1081-4782-856f-d7e6d8b8b927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:49.346184 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5510a-1081-4782-856f-d7e6d8b8b927 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:49.346865 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded5510a-1081-4782-856f-d7e6d8b8b927 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:49.348772 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded5510a-1081-4782-856f-d7e6d8b8b927 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:49.373248 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 967/1917] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:03:49 2022] POST /identity/v3/projects => generated 383 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:49.377184 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd0c949-213a-4340-96a7-557cffc34856 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:49.388278 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd0c949-213a-4340-96a7-557cffc34856 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:49.446920 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cff23b4-937c-4875-a179-d480f6d0314d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:49.449062 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff23b4-937c-4875-a179-d480f6d0314d admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:49.449348 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff23b4-937c-4875-a179-d480f6d0314d admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:49.449626 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff23b4-937c-4875-a179-d480f6d0314d admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:49.450355 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cff23b4-937c-4875-a179-d480f6d0314d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:49.452323 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cff23b4-937c-4875-a179-d480f6d0314d admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:49.473542 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 951/1918] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:03:49 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:03:49.477808 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded5510a-1081-4782-856f-d7e6d8b8b927 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:49.492384 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded5510a-1081-4782-856f-d7e6d8b8b927 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:49.497220 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7de6b3-541b-48a3-9310-429ea21bc0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:49.497914 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7de6b3-541b-48a3-9310-429ea21bc0a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:49.498255 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7de6b3-541b-48a3-9310-429ea21bc0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:49.498534 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7de6b3-541b-48a3-9310-429ea21bc0a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:49.499425 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7de6b3-541b-48a3-9310-429ea21bc0a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:49.501016 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7de6b3-541b-48a3-9310-429ea21bc0a0 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:49.507220 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:49.517070 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 968/1919] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:49 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:49.518444 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cff23b4-937c-4875-a179-d480f6d0314d admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:49.529260 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cff23b4-937c-4875-a179-d480f6d0314d admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:49.534549 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79407f3c-3e77-4f29-bbe7-bf183fda7f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:49.535599 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79407f3c-3e77-4f29-bbe7-bf183fda7f4f admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:49.535916 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79407f3c-3e77-4f29-bbe7-bf183fda7f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:49.536296 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79407f3c-3e77-4f29-bbe7-bf183fda7f4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:49.537378 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79407f3c-3e77-4f29-bbe7-bf183fda7f4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:49.539756 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79407f3c-3e77-4f29-bbe7-bf183fda7f4f admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:49.550593 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 952/1920] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:49.554931 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7de6b3-541b-48a3-9310-429ea21bc0a0 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:49.565850 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7de6b3-541b-48a3-9310-429ea21bc0a0 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:49.570687 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc35187-36a8-4399-95b5-dd1b61a35d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:49.571641 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc35187-36a8-4399-95b5-dd1b61a35d3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:49.571887 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc35187-36a8-4399-95b5-dd1b61a35d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:49.572161 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc35187-36a8-4399-95b5-dd1b61a35d3c admin admin] PATH_INFO: `/v3/projects/8c9f461e575c4b0b84271fd61593cb62/users/0d119e3f1567445ca4bf46d1c71e1a19/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:49.587822 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc35187-36a8-4399-95b5-dd1b61a35d3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c9f461e575c4b0b84271fd61593cb62, user_id=0d119e3f1567445ca4bf46d1c71e1a19, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:49.591398 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc35187-36a8-4399-95b5-dd1b61a35d3c admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:49.605360 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 969/1921] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:03:49 2022] PUT /identity/v3/projects/8c9f461e575c4b0b84271fd61593cb62/users/0d119e3f1567445ca4bf46d1c71e1a19/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:49.616782 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504ec54-1b5a-4491-9e39-ce5613bab907 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:49.617076 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504ec54-1b5a-4491-9e39-ce5613bab907 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:49.617358 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504ec54-1b5a-4491-9e39-ce5613bab907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:49.644649 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-d504ec54-1b5a-4491-9e39-ce5613bab907 None None] MFA Rules not processed for user `0d119e3f1567445ca4bf46d1c71e1a19`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:49.645885 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-d504ec54-1b5a-4491-9e39-ce5613bab907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:49.711428 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 953/1922] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:49.716232 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc35187-36a8-4399-95b5-dd1b61a35d3c admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:49.782274 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc35187-36a8-4399-95b5-dd1b61a35d3c admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:49.787638 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a83705-5721-4d0f-b4f2-d61084ad1f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:49.788305 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a83705-5721-4d0f-b4f2-d61084ad1f2c admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:49.788591 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a83705-5721-4d0f-b4f2-d61084ad1f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:49.788869 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a83705-5721-4d0f-b4f2-d61084ad1f2c admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:49.789565 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a83705-5721-4d0f-b4f2-d61084ad1f2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:49.791791 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a83705-5721-4d0f-b4f2-d61084ad1f2c admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:49.818590 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 970/1923] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:03:49 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:03:49.822883 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d504ec54-1b5a-4491-9e39-ce5613bab907 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:49.834594 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d504ec54-1b5a-4491-9e39-ce5613bab907 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:49.884647 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:49.892231 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19a770f-bbdf-48f6-9ba7-574ed6cfa2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:49.892897 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a770f-bbdf-48f6-9ba7-574ed6cfa2a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:49.893178 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a770f-bbdf-48f6-9ba7-574ed6cfa2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:49.893451 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a770f-bbdf-48f6-9ba7-574ed6cfa2a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:49.894136 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19a770f-bbdf-48f6-9ba7-574ed6cfa2a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:49.896081 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19a770f-bbdf-48f6-9ba7-574ed6cfa2a1 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:49.915195 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 954/1924] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:03:49 2022] POST /identity/v3/users => generated 299 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:03:49.919893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a83705-5721-4d0f-b4f2-d61084ad1f2c admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:49.929866 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a83705-5721-4d0f-b4f2-d61084ad1f2c admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:49.934591 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ce159-c4fc-44be-a45d-0177f3be31bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:49.935251 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ce159-c4fc-44be-a45d-0177f3be31bd admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:49.935560 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ce159-c4fc-44be-a45d-0177f3be31bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:49.935834 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ce159-c4fc-44be-a45d-0177f3be31bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:49.936555 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ce159-c4fc-44be-a45d-0177f3be31bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:49.938139 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ce159-c4fc-44be-a45d-0177f3be31bd admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:49.948838 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 971/1925] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:49.953002 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19a770f-bbdf-48f6-9ba7-574ed6cfa2a1 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:49.963000 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19a770f-bbdf-48f6-9ba7-574ed6cfa2a1 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:49.968450 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbb27c3-6855-4782-b013-d97029edd9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:49.969356 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbb27c3-6855-4782-b013-d97029edd9aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:49.969648 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbb27c3-6855-4782-b013-d97029edd9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:49.969948 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbb27c3-6855-4782-b013-d97029edd9aa admin admin] PATH_INFO: `/v3/projects/18b7ee947b1e47adbc5cdb79d8ab2439/users/1960724ba28a478d8cc317343f18e8db/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:49.985747 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbb27c3-6855-4782-b013-d97029edd9aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=18b7ee947b1e47adbc5cdb79d8ab2439, user_id=1960724ba28a478d8cc317343f18e8db, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:49.988554 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbb27c3-6855-4782-b013-d97029edd9aa admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:50.001964 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 955/1926] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:03:49 2022] PUT /identity/v3/projects/18b7ee947b1e47adbc5cdb79d8ab2439/users/1960724ba28a478d8cc317343f18e8db/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:50.006101 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ce159-c4fc-44be-a45d-0177f3be31bd admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:50.016641 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ce159-c4fc-44be-a45d-0177f3be31bd admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:50.073263 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd7792-78f6-409a-9bdd-011e94af82c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:50.074094 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd7792-78f6-409a-9bdd-011e94af82c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:50.074375 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd7792-78f6-409a-9bdd-011e94af82c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:50.074644 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd7792-78f6-409a-9bdd-011e94af82c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:50.075393 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfd7792-78f6-409a-9bdd-011e94af82c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:50.076975 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfd7792-78f6-409a-9bdd-011e94af82c7 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:50.086548 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 972/1927] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:03:50 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:03:50.091345 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbb27c3-6855-4782-b013-d97029edd9aa admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:50.102047 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbb27c3-6855-4782-b013-d97029edd9aa admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:50.107145 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e2c8b-827b-459f-a0ad-b2f5ab37db92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:50.108236 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e2c8b-827b-459f-a0ad-b2f5ab37db92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:50.108545 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e2c8b-827b-459f-a0ad-b2f5ab37db92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:50.108851 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e2c8b-827b-459f-a0ad-b2f5ab37db92 admin admin] PATH_INFO: `/v3/projects/18b7ee947b1e47adbc5cdb79d8ab2439/users/1960724ba28a478d8cc317343f18e8db/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:50.110998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5e2c8b-827b-459f-a0ad-b2f5ab37db92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18b7ee947b1e47adbc5cdb79d8ab2439, user_id=1960724ba28a478d8cc317343f18e8db, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:50.113722 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5e2c8b-827b-459f-a0ad-b2f5ab37db92 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:50.124440 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 956/1928] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:03:50 2022] PUT /identity/v3/projects/18b7ee947b1e47adbc5cdb79d8ab2439/users/1960724ba28a478d8cc317343f18e8db/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:03:50.134448 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635f512-72ba-4945-98d7-df6e90d6e203 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:50.134795 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635f512-72ba-4945-98d7-df6e90d6e203 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:50.135773 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635f512-72ba-4945-98d7-df6e90d6e203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:50.163522 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-5635f512-72ba-4945-98d7-df6e90d6e203 None None] MFA Rules not processed for user `1960724ba28a478d8cc317343f18e8db`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:50.164920 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-5635f512-72ba-4945-98d7-df6e90d6e203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:50.208373 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:50.254470 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 973/1929] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:03:50 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:03:50.267092 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e2c8b-827b-459f-a0ad-b2f5ab37db92 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:50.268945 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-7f5e2c8b-827b-459f-a0ad-b2f5ab37db92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:50.330475 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e2c8b-827b-459f-a0ad-b2f5ab37db92 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:50.334146 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e9b90-e4c2-4e3c-8250-4a80a516fbd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:50.334799 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e9b90-e4c2-4e3c-8250-4a80a516fbd5 service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:50.335085 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e9b90-e4c2-4e3c-8250-4a80a516fbd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:50.335437 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e9b90-e4c2-4e3c-8250-4a80a516fbd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:50.402676 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5e9b90-e4c2-4e3c-8250-4a80a516fbd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:50.404209 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5e9b90-e4c2-4e3c-8250-4a80a516fbd5 service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:50.412911 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 957/1930] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:03:53.070139 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5635f512-72ba-4945-98d7-df6e90d6e203 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:53.071579 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-5635f512-72ba-4945-98d7-df6e90d6e203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:53.090236 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5635f512-72ba-4945-98d7-df6e90d6e203 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:53.095261 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48711825-0a6b-4d91-ad08-cedb322980e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:53.096153 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48711825-0a6b-4d91-ad08-cedb322980e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:53.096586 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48711825-0a6b-4d91-ad08-cedb322980e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:53.097010 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48711825-0a6b-4d91-ad08-cedb322980e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:53.169949 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48711825-0a6b-4d91-ad08-cedb322980e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:53.171470 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48711825-0a6b-4d91-ad08-cedb322980e5 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:53.195483 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 974/1931] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:03:54.116929 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e9b90-e4c2-4e3c-8250-4a80a516fbd5 service cinder] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:54.129860 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e9b90-e4c2-4e3c-8250-4a80a516fbd5 service cinder] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:54.135567 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58a7619-18fd-402a-91eb-5cadfede89fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:54.136391 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a7619-18fd-402a-91eb-5cadfede89fe service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:54.136860 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a7619-18fd-402a-91eb-5cadfede89fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:54.137188 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a7619-18fd-402a-91eb-5cadfede89fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:54.149870 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58a7619-18fd-402a-91eb-5cadfede89fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:54.151736 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58a7619-18fd-402a-91eb-5cadfede89fe service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:54.162293 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 958/1932] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:54 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:03:54.511451 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea72d552-5bf0-43cc-a719-fb4a16fd046e None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:54.511918 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea72d552-5bf0-43cc-a719-fb4a16fd046e None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:54.512003 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea72d552-5bf0-43cc-a719-fb4a16fd046e None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:54.513487 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 975/1933] 10.222.0.13 () {54 vars in 921 bytes} [Tue Jun 7 22:03:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 22:03:54.521620 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc104e-795b-4608-b459-7b68918886c1 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:54.521919 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc104e-795b-4608-b459-7b68918886c1 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:54.522208 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc104e-795b-4608-b459-7b68918886c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:54.552013 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-8cbc104e-795b-4608-b459-7b68918886c1 None None] MFA Rules not processed for user `dbe0303ea0764347bfb054870ebba41f`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:03:54.564056 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-8cbc104e-795b-4608-b459-7b68918886c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:54.672918 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:03:54.710909 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 959/1934] 10.222.0.13 () {58 vars in 1018 bytes} [Tue Jun 7 22:03:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:03:54.726983 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea72d552-5bf0-43cc-a719-fb4a16fd046e None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:03:54.730415 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-ea72d552-5bf0-43cc-a719-fb4a16fd046e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:03:54.753167 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea72d552-5bf0-43cc-a719-fb4a16fd046e None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:03:54.772016 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7cd9af-28e4-4a64-b06d-8736939fab3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:03:54.773018 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7cd9af-28e4-4a64-b06d-8736939fab3d service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:03:54.773303 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7cd9af-28e4-4a64-b06d-8736939fab3d service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:03:54.774586 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7cd9af-28e4-4a64-b06d-8736939fab3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:03:54.788633 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7cd9af-28e4-4a64-b06d-8736939fab3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:03:54.790906 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7cd9af-28e4-4a64-b06d-8736939fab3d service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:03:54.812482 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 976/1935] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:03:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 22:04:00.428845 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc104e-795b-4608-b459-7b68918886c1 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:00.430279 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-8cbc104e-795b-4608-b459-7b68918886c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:00.518878 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc104e-795b-4608-b459-7b68918886c1 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:00.524679 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc588eb8-5d38-459f-902a-97f7f62ba2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:00.525622 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc588eb8-5d38-459f-902a-97f7f62ba2d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:00.526714 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc588eb8-5d38-459f-902a-97f7f62ba2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:00.527072 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc588eb8-5d38-459f-902a-97f7f62ba2d2 admin admin] PATH_INFO: `/v3/users/e5775e2dab8244edaacb1440323fe111` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:00.543977 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc588eb8-5d38-459f-902a-97f7f62ba2d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5775e2dab8244edaacb1440323fe111)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:00.548716 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc588eb8-5d38-459f-902a-97f7f62ba2d2 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:00.610176 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-dc588eb8-5d38-459f-902a-97f7f62ba2d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5775e2dab8244edaacb1440323fe111'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:00.611015 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-dc588eb8-5d38-459f-902a-97f7f62ba2d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5775e2dab8244edaacb1440323fe111'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:00.623026 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-dc588eb8-5d38-459f-902a-97f7f62ba2d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5775e2dab8244edaacb1440323fe111'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:00.644957 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-dc588eb8-5d38-459f-902a-97f7f62ba2d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5775e2dab8244edaacb1440323fe111'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:00.655195 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-dc588eb8-5d38-459f-902a-97f7f62ba2d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5775e2dab8244edaacb1440323fe111'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:00.666925 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 960/1936] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:04:00 2022] DELETE /identity/v3/users/e5775e2dab8244edaacb1440323fe111 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:04:00.668610 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37af9e1-ef1c-4ec7-8a95-687095c92962 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:00.669586 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37af9e1-ef1c-4ec7-8a95-687095c92962 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:00.670032 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37af9e1-ef1c-4ec7-8a95-687095c92962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:00.694050 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-f37af9e1-ef1c-4ec7-8a95-687095c92962 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:04:00.696704 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-f37af9e1-ef1c-4ec7-8a95-687095c92962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:00.836131 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 977/1937] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:04:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:04:00.857002 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc588eb8-5d38-459f-902a-97f7f62ba2d2 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:00.858845 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-dc588eb8-5d38-459f-902a-97f7f62ba2d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:01.033035 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc588eb8-5d38-459f-902a-97f7f62ba2d2 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:01.041377 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda9e7c-40e1-4c54-9618-d22e0e6c8e61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:01.045701 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda9e7c-40e1-4c54-9618-d22e0e6c8e61 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:01.047118 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda9e7c-40e1-4c54-9618-d22e0e6c8e61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:01.047579 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda9e7c-40e1-4c54-9618-d22e0e6c8e61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:01.073270 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdda9e7c-40e1-4c54-9618-d22e0e6c8e61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:01.076472 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdda9e7c-40e1-4c54-9618-d22e0e6c8e61 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:01.090207 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 961/1938] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:04:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:04:01.696640 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37af9e1-ef1c-4ec7-8a95-687095c92962 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:01.699465 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-f37af9e1-ef1c-4ec7-8a95-687095c92962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:01.713324 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37af9e1-ef1c-4ec7-8a95-687095c92962 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:01.718778 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544828d7-35f9-4168-9964-c57feba8da6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:01.719863 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544828d7-35f9-4168-9964-c57feba8da6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:01.720203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544828d7-35f9-4168-9964-c57feba8da6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:01.720630 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544828d7-35f9-4168-9964-c57feba8da6b admin admin] PATH_INFO: `/v3/projects/365637613bba4e26bf79d0e5dc24a2d1` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:01.721827 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544828d7-35f9-4168-9964-c57feba8da6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=365637613bba4e26bf79d0e5dc24a2d1)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:01.724089 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544828d7-35f9-4168-9964-c57feba8da6b admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:01.787956 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-544828d7-35f9-4168-9964-c57feba8da6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '365637613bba4e26bf79d0e5dc24a2d1', 'request_id': 'req-544828d7-35f9-4168-9964-c57feba8da6b'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:01.791083 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:04:01.798121 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-544828d7-35f9-4168-9964-c57feba8da6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '365637613bba4e26bf79d0e5dc24a2d1', 'request_id': 'req-544828d7-35f9-4168-9964-c57feba8da6b'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:01.805137 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-544828d7-35f9-4168-9964-c57feba8da6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '365637613bba4e26bf79d0e5dc24a2d1', 'request_id': 'req-544828d7-35f9-4168-9964-c57feba8da6b'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:01.817049 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-544828d7-35f9-4168-9964-c57feba8da6b admin admin] The token cache is being invalidate because project 365637613bba4e26bf79d0e5dc24a2d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:04:01.817335 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-544828d7-35f9-4168-9964-c57feba8da6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:01.818213 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-544828d7-35f9-4168-9964-c57feba8da6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:01.819777 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 978/1939] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:04:01 2022] DELETE /identity/v3/projects/365637613bba4e26bf79d0e5dc24a2d1 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:04:01.824570 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda9e7c-40e1-4c54-9618-d22e0e6c8e61 service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:01.829483 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:04:01.836243 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda9e7c-40e1-4c54-9618-d22e0e6c8e61 service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:01.902925 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded185fd-b543-4feb-a5b4-47547d346522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:01.903833 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded185fd-b543-4feb-a5b4-47547d346522 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:01.904172 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded185fd-b543-4feb-a5b4-47547d346522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:01.904450 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded185fd-b543-4feb-a5b4-47547d346522 admin admin] PATH_INFO: `/v3/users/8659aafcb2784b13ba5428f0a70be7d6` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:01.905648 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded185fd-b543-4feb-a5b4-47547d346522 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8659aafcb2784b13ba5428f0a70be7d6)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:01.907603 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded185fd-b543-4feb-a5b4-47547d346522 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:01.956582 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-ded185fd-b543-4feb-a5b4-47547d346522 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8659aafcb2784b13ba5428f0a70be7d6'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:01.957221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-ded185fd-b543-4feb-a5b4-47547d346522 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8659aafcb2784b13ba5428f0a70be7d6'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:01.964504 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-ded185fd-b543-4feb-a5b4-47547d346522 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8659aafcb2784b13ba5428f0a70be7d6'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:01.978309 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-ded185fd-b543-4feb-a5b4-47547d346522 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8659aafcb2784b13ba5428f0a70be7d6'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:01.985779 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-ded185fd-b543-4feb-a5b4-47547d346522 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8659aafcb2784b13ba5428f0a70be7d6'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:01.992845 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 962/1940] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:04:01 2022] DELETE /identity/v3/users/8659aafcb2784b13ba5428f0a70be7d6 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:04:03.168896 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544828d7-35f9-4168-9964-c57feba8da6b admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:03.170389 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-544828d7-35f9-4168-9964-c57feba8da6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:03.191961 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544828d7-35f9-4168-9964-c57feba8da6b admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:03.293261 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a4c2ef-cefc-4d42-9a5d-c844e01970a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:03.294141 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a4c2ef-cefc-4d42-9a5d-c844e01970a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:03.294475 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a4c2ef-cefc-4d42-9a5d-c844e01970a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:03.294755 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a4c2ef-cefc-4d42-9a5d-c844e01970a9 admin admin] PATH_INFO: `/v3/projects/4280a5f3e53c41178100e3d90d49aa66` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:03.296052 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a4c2ef-cefc-4d42-9a5d-c844e01970a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4280a5f3e53c41178100e3d90d49aa66)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:03.297970 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a4c2ef-cefc-4d42-9a5d-c844e01970a9 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:03.348136 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a4a4c2ef-cefc-4d42-9a5d-c844e01970a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4280a5f3e53c41178100e3d90d49aa66', 'request_id': 'req-a4a4c2ef-cefc-4d42-9a5d-c844e01970a9'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:03.359900 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a4a4c2ef-cefc-4d42-9a5d-c844e01970a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4280a5f3e53c41178100e3d90d49aa66', 'request_id': 'req-a4a4c2ef-cefc-4d42-9a5d-c844e01970a9'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:03.369239 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a4a4c2ef-cefc-4d42-9a5d-c844e01970a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4280a5f3e53c41178100e3d90d49aa66', 'request_id': 'req-a4a4c2ef-cefc-4d42-9a5d-c844e01970a9'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:03.381373 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a4a4c2ef-cefc-4d42-9a5d-c844e01970a9 admin admin] The token cache is being invalidate because project 4280a5f3e53c41178100e3d90d49aa66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:04:03.381675 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a4a4c2ef-cefc-4d42-9a5d-c844e01970a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:03.382407 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a4a4c2ef-cefc-4d42-9a5d-c844e01970a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:03.391556 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 979/1941] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:04:03 2022] DELETE /identity/v3/projects/4280a5f3e53c41178100e3d90d49aa66 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:04:03.393234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f309b-20c1-41a9-bfe3-ed7a803abdf3 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:03.393678 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f309b-20c1-41a9-bfe3-ed7a803abdf3 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:03.393819 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f309b-20c1-41a9-bfe3-ed7a803abdf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:03.413091 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-ef3f309b-20c1-41a9-bfe3-ed7a803abdf3 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:04:03.414375 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-ef3f309b-20c1-41a9-bfe3-ed7a803abdf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:03.480631 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 963/1942] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:04:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:04:03.488825 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8e88d-b5dd-4f0e-8c19-880fd7b1e37e None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:03.489134 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8e88d-b5dd-4f0e-8c19-880fd7b1e37e None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:03.489702 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8e88d-b5dd-4f0e-8c19-880fd7b1e37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:03.511507 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-1ce8e88d-b5dd-4f0e-8c19-880fd7b1e37e None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:04:03.512846 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-1ce8e88d-b5dd-4f0e-8c19-880fd7b1e37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:03.521055 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 980/1943] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:04:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:04:03.525583 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3f309b-20c1-41a9-bfe3-ed7a803abdf3 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:03.535953 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3f309b-20c1-41a9-bfe3-ed7a803abdf3 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:03.542626 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fe8263-44c9-4d9d-8f51-160247a37dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:03.543504 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe8263-44c9-4d9d-8f51-160247a37dec admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:03.543796 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe8263-44c9-4d9d-8f51-160247a37dec admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:03.544137 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe8263-44c9-4d9d-8f51-160247a37dec admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:03.544868 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fe8263-44c9-4d9d-8f51-160247a37dec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:04:03.545186 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fe8263-44c9-4d9d-8f51-160247a37dec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:03.546867 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fe8263-44c9-4d9d-8f51-160247a37dec admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:03.564094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 964/1944] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:04:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:04:03.567198 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8e88d-b5dd-4f0e-8c19-880fd7b1e37e None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:03.578672 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8e88d-b5dd-4f0e-8c19-880fd7b1e37e None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:03.587622 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84fe34f-9717-4559-b900-286f5643d7b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:03.588323 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84fe34f-9717-4559-b900-286f5643d7b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:03.588687 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84fe34f-9717-4559-b900-286f5643d7b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:03.589212 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84fe34f-9717-4559-b900-286f5643d7b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:03.590084 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84fe34f-9717-4559-b900-286f5643d7b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:03.592173 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84fe34f-9717-4559-b900-286f5643d7b8 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:03.634001 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 981/1945] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:04:03 2022] POST /identity/v3/projects => generated 377 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:04:03.638779 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fe8263-44c9-4d9d-8f51-160247a37dec admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:03.649033 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fe8263-44c9-4d9d-8f51-160247a37dec admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:03.654584 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:04:03.722196 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f7891c-2464-4c14-92fb-15c79104adb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:03.722908 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7891c-2464-4c14-92fb-15c79104adb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:03.723156 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7891c-2464-4c14-92fb-15c79104adb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:03.723479 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7891c-2464-4c14-92fb-15c79104adb2 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:03.724214 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f7891c-2464-4c14-92fb-15c79104adb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:03.726769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f7891c-2464-4c14-92fb-15c79104adb2 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:03.731757 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84fe34f-9717-4559-b900-286f5643d7b8 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:03.733619 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-b84fe34f-9717-4559-b900-286f5643d7b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:03.748219 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84fe34f-9717-4559-b900-286f5643d7b8 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:03.753288 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 965/1946] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:04:03 2022] POST /identity/v3/users => generated 289 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:04:03.756274 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6fc1ec-ef49-4514-8654-92cd6cb8efc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:03.757193 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6fc1ec-ef49-4514-8654-92cd6cb8efc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:03.757478 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6fc1ec-ef49-4514-8654-92cd6cb8efc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:03.757757 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6fc1ec-ef49-4514-8654-92cd6cb8efc3 admin admin] PATH_INFO: `/v3/users/7bb7c70d094149fd9c3234e9c237faec` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:03.759042 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6fc1ec-ef49-4514-8654-92cd6cb8efc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bb7c70d094149fd9c3234e9c237faec)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:03.761085 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f7891c-2464-4c14-92fb-15c79104adb2 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:03.761995 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6fc1ec-ef49-4514-8654-92cd6cb8efc3 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:03.779118 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f7891c-2464-4c14-92fb-15c79104adb2 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:03.784982 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ff1e1-7a52-440a-93e9-bdc58e9b28e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:03.785688 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ff1e1-7a52-440a-93e9-bdc58e9b28e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:03.785975 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ff1e1-7a52-440a-93e9-bdc58e9b28e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:03.786250 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ff1e1-7a52-440a-93e9-bdc58e9b28e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:03.786987 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896ff1e1-7a52-440a-93e9-bdc58e9b28e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:03.789066 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896ff1e1-7a52-440a-93e9-bdc58e9b28e0 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:03.802705 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 966/1947] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:04:03 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:04:03.806823 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ff1e1-7a52-440a-93e9-bdc58e9b28e0 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:03.816915 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ff1e1-7a52-440a-93e9-bdc58e9b28e0 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:03.821748 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2006cda-f8c8-4f1f-ac6f-1523b42a9bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:03.821913 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:04:03.822399 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2006cda-f8c8-4f1f-ac6f-1523b42a9bee admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:03.823400 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2006cda-f8c8-4f1f-ac6f-1523b42a9bee admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:03.823822 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2006cda-f8c8-4f1f-ac6f-1523b42a9bee admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:03.824575 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2006cda-f8c8-4f1f-ac6f-1523b42a9bee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:03.826227 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2006cda-f8c8-4f1f-ac6f-1523b42a9bee admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:03.836460 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-eb6fc1ec-ef49-4514-8654-92cd6cb8efc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bb7c70d094149fd9c3234e9c237faec'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:03.837942 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 967/1948] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:04:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 22:04:03.842575 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2006cda-f8c8-4f1f-ac6f-1523b42a9bee admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:03.848968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-eb6fc1ec-ef49-4514-8654-92cd6cb8efc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bb7c70d094149fd9c3234e9c237faec'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:03.853472 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2006cda-f8c8-4f1f-ac6f-1523b42a9bee admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:03.859227 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7c6b8a-12b6-4488-ac39-dd19e163be17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:03.860273 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c6b8a-12b6-4488-ac39-dd19e163be17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:03.860565 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c6b8a-12b6-4488-ac39-dd19e163be17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:03.860838 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c6b8a-12b6-4488-ac39-dd19e163be17 admin admin] PATH_INFO: `/v3/projects/02bb43fcee824adf8e60dfffcc5258c0/users/42b3410fb02f44748ce4afbf3e911a45/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:03.862948 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-eb6fc1ec-ef49-4514-8654-92cd6cb8efc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bb7c70d094149fd9c3234e9c237faec'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:03.863783 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-eb6fc1ec-ef49-4514-8654-92cd6cb8efc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bb7c70d094149fd9c3234e9c237faec'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:03.865451 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-eb6fc1ec-ef49-4514-8654-92cd6cb8efc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bb7c70d094149fd9c3234e9c237faec'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:03.877779 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7c6b8a-12b6-4488-ac39-dd19e163be17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02bb43fcee824adf8e60dfffcc5258c0, user_id=42b3410fb02f44748ce4afbf3e911a45, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:03.880424 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 982/1949] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:04:03 2022] DELETE /identity/v3/users/7bb7c70d094149fd9c3234e9c237faec => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:04:03.883553 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7c6b8a-12b6-4488-ac39-dd19e163be17 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:03.884473 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd33c4-fdac-4ef7-9968-d58ccb2aef21 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:03.884768 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd33c4-fdac-4ef7-9968-d58ccb2aef21 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:03.885702 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd33c4-fdac-4ef7-9968-d58ccb2aef21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:03.900026 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 968/1950] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:04:03 2022] PUT /identity/v3/projects/02bb43fcee824adf8e60dfffcc5258c0/users/42b3410fb02f44748ce4afbf3e911a45/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:04:03.905658 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-7ebd33c4-fdac-4ef7-9968-d58ccb2aef21 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:04:03.906865 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-7ebd33c4-fdac-4ef7-9968-d58ccb2aef21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:03.916565 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41655d15-5deb-476a-870d-1b49accc4eff None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:03.917004 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41655d15-5deb-476a-870d-1b49accc4eff None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:03.917361 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41655d15-5deb-476a-870d-1b49accc4eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:03.944830 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-41655d15-5deb-476a-870d-1b49accc4eff None None] MFA Rules not processed for user `42b3410fb02f44748ce4afbf3e911a45`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:04:03.946086 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-41655d15-5deb-476a-870d-1b49accc4eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:03.973693 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 983/1951] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:04:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:04:03.983793 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd33c4-fdac-4ef7-9968-d58ccb2aef21 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:03.985361 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-7ebd33c4-fdac-4ef7-9968-d58ccb2aef21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:04.045938 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 969/1952] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:04:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:04:04.058615 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41655d15-5deb-476a-870d-1b49accc4eff None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:04.060785 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-41655d15-5deb-476a-870d-1b49accc4eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:04.075084 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd33c4-fdac-4ef7-9968-d58ccb2aef21 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:04.079597 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab2bb0-8463-42ad-943a-29248fd937f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:04.081007 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab2bb0-8463-42ad-943a-29248fd937f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:04.081307 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab2bb0-8463-42ad-943a-29248fd937f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:04.081590 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab2bb0-8463-42ad-943a-29248fd937f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:04.092854 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cab2bb0-8463-42ad-943a-29248fd937f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:04.094313 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cab2bb0-8463-42ad-943a-29248fd937f8 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:04.106056 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 984/1953] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:04:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:04:04.123547 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:04:04.141433 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41655d15-5deb-476a-870d-1b49accc4eff None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:04.145994 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b530064a-e3f1-4f2b-afcb-d751d4f83f0f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96987b04471944bc86ab44ff9936c732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:04.146695 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530064a-e3f1-4f2b-afcb-d751d4f83f0f service glance] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:04.146928 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530064a-e3f1-4f2b-afcb-d751d4f83f0f service glance] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:04.147204 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530064a-e3f1-4f2b-afcb-d751d4f83f0f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:04.157022 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b530064a-e3f1-4f2b-afcb-d751d4f83f0f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:04.158452 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b530064a-e3f1-4f2b-afcb-d751d4f83f0f service glance] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:04.168348 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 970/1954] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:04:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 22:04:04.684247 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab2bb0-8463-42ad-943a-29248fd937f8 service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:04.686435 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-3cab2bb0-8463-42ad-943a-29248fd937f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:04.704342 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab2bb0-8463-42ad-943a-29248fd937f8 service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:04.710927 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145bdc19-b163-4700-9e29-f4d07b7bdb62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:04.712317 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145bdc19-b163-4700-9e29-f4d07b7bdb62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:04.713914 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145bdc19-b163-4700-9e29-f4d07b7bdb62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:04.714262 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145bdc19-b163-4700-9e29-f4d07b7bdb62 admin admin] PATH_INFO: `/v3/projects/fec79ef616644f82ba670e3962dd0e4c` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:04.716166 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145bdc19-b163-4700-9e29-f4d07b7bdb62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fec79ef616644f82ba670e3962dd0e4c)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:04.719167 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145bdc19-b163-4700-9e29-f4d07b7bdb62 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:04.784893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-145bdc19-b163-4700-9e29-f4d07b7bdb62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fec79ef616644f82ba670e3962dd0e4c', 'request_id': 'req-145bdc19-b163-4700-9e29-f4d07b7bdb62'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:04.788431 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:04:04.800799 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-145bdc19-b163-4700-9e29-f4d07b7bdb62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fec79ef616644f82ba670e3962dd0e4c', 'request_id': 'req-145bdc19-b163-4700-9e29-f4d07b7bdb62'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:04.813194 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-145bdc19-b163-4700-9e29-f4d07b7bdb62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fec79ef616644f82ba670e3962dd0e4c', 'request_id': 'req-145bdc19-b163-4700-9e29-f4d07b7bdb62'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:04.831002 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-145bdc19-b163-4700-9e29-f4d07b7bdb62 admin admin] The token cache is being invalidate because project fec79ef616644f82ba670e3962dd0e4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:04:04.831002 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-145bdc19-b163-4700-9e29-f4d07b7bdb62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:04.831629 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-145bdc19-b163-4700-9e29-f4d07b7bdb62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:04.832297 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b530064a-e3f1-4f2b-afcb-d751d4f83f0f service glance] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:04.833336 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 985/1955] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:04:04 2022] DELETE /identity/v3/projects/fec79ef616644f82ba670e3962dd0e4c => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:04:04.833954 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-b530064a-e3f1-4f2b-afcb-d751d4f83f0f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:04.837052 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145bdc19-b163-4700-9e29-f4d07b7bdb62 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:04.838917 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-145bdc19-b163-4700-9e29-f4d07b7bdb62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:04.860106 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b530064a-e3f1-4f2b-afcb-d751d4f83f0f service glance] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:04.957648 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145bdc19-b163-4700-9e29-f4d07b7bdb62 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:04.965369 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d4549-c6f0-4f79-9848-dd577cd4a3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:04.966257 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d4549-c6f0-4f79-9848-dd577cd4a3fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:04.966585 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d4549-c6f0-4f79-9848-dd577cd4a3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:04.966860 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d4549-c6f0-4f79-9848-dd577cd4a3fc admin admin] PATH_INFO: `/v3/users/695f4a75ceb74be0923ab8a1a30fb76b` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:04.968138 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7d4549-c6f0-4f79-9848-dd577cd4a3fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=695f4a75ceb74be0923ab8a1a30fb76b)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:04.970268 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7d4549-c6f0-4f79-9848-dd577cd4a3fc admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:04.977297 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b38ffd5-3de4-4270-9371-c5214c1571f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:04.978381 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b38ffd5-3de4-4270-9371-c5214c1571f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:04.979515 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b38ffd5-3de4-4270-9371-c5214c1571f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:04.979999 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b38ffd5-3de4-4270-9371-c5214c1571f7 admin admin] PATH_INFO: `/v3/users/2d3c69e4fd6148f295369531caf98b65` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:04.982069 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b38ffd5-3de4-4270-9371-c5214c1571f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d3c69e4fd6148f295369531caf98b65)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:04.983989 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b38ffd5-3de4-4270-9371-c5214c1571f7 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:05.046628 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-9e7d4549-c6f0-4f79-9848-dd577cd4a3fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '695f4a75ceb74be0923ab8a1a30fb76b'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:05.050991 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-0b38ffd5-3de4-4270-9371-c5214c1571f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d3c69e4fd6148f295369531caf98b65'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:05.051735 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-0b38ffd5-3de4-4270-9371-c5214c1571f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d3c69e4fd6148f295369531caf98b65'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:05.055021 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-9e7d4549-c6f0-4f79-9848-dd577cd4a3fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '695f4a75ceb74be0923ab8a1a30fb76b'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:05.060438 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-0b38ffd5-3de4-4270-9371-c5214c1571f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d3c69e4fd6148f295369531caf98b65'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:05.070669 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-9e7d4549-c6f0-4f79-9848-dd577cd4a3fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '695f4a75ceb74be0923ab8a1a30fb76b'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:05.071518 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-9e7d4549-c6f0-4f79-9848-dd577cd4a3fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '695f4a75ceb74be0923ab8a1a30fb76b'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:05.072212 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-9e7d4549-c6f0-4f79-9848-dd577cd4a3fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '695f4a75ceb74be0923ab8a1a30fb76b'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:05.076527 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-0b38ffd5-3de4-4270-9371-c5214c1571f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d3c69e4fd6148f295369531caf98b65'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:05.085686 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-0b38ffd5-3de4-4270-9371-c5214c1571f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d3c69e4fd6148f295369531caf98b65'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:05.089934 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 986/1956] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:04:04 2022] DELETE /identity/v3/users/695f4a75ceb74be0923ab8a1a30fb76b => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:04:05.096420 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 971/1957] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:04:04 2022] DELETE /identity/v3/users/2d3c69e4fd6148f295369531caf98b65 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:04:05.101065 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c3f5c7-e17c-4af2-a055-126d4fd66b7d None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:05.101974 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c3f5c7-e17c-4af2-a055-126d4fd66b7d None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:05.101974 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c3f5c7-e17c-4af2-a055-126d4fd66b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:05.130902 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-a1c3f5c7-e17c-4af2-a055-126d4fd66b7d None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:04:05.132367 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a1c3f5c7-e17c-4af2-a055-126d4fd66b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:05.206476 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:04:05.217674 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 987/1958] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:04:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:04:05.227948 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b38ffd5-3de4-4270-9371-c5214c1571f7 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:05.230132 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-0b38ffd5-3de4-4270-9371-c5214c1571f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:05.323259 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b38ffd5-3de4-4270-9371-c5214c1571f7 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:05.328226 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497faa49-c8aa-4063-bb54-a146fefacf24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:05.329008 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497faa49-c8aa-4063-bb54-a146fefacf24 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:05.329388 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497faa49-c8aa-4063-bb54-a146fefacf24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:05.330261 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497faa49-c8aa-4063-bb54-a146fefacf24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:05.342607 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497faa49-c8aa-4063-bb54-a146fefacf24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:05.344141 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497faa49-c8aa-4063-bb54-a146fefacf24 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:05.357155 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 972/1959] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:04:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 22:04:05.437906 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c3f5c7-e17c-4af2-a055-126d4fd66b7d None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:05.441521 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a1c3f5c7-e17c-4af2-a055-126d4fd66b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:05.478084 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c3f5c7-e17c-4af2-a055-126d4fd66b7d None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:05.483785 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d6d187-b5c8-44a9-bccf-af998cfad909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:05.484771 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d6d187-b5c8-44a9-bccf-af998cfad909 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:05.485265 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d6d187-b5c8-44a9-bccf-af998cfad909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:05.485487 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d6d187-b5c8-44a9-bccf-af998cfad909 admin admin] PATH_INFO: `/v3/projects/c393b6a3ec3141c3aff392278b481b31` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:05.486882 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d6d187-b5c8-44a9-bccf-af998cfad909 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c393b6a3ec3141c3aff392278b481b31)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:05.488896 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d6d187-b5c8-44a9-bccf-af998cfad909 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:05.546448 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a7d6d187-b5c8-44a9-bccf-af998cfad909 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c393b6a3ec3141c3aff392278b481b31', 'request_id': 'req-a7d6d187-b5c8-44a9-bccf-af998cfad909'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:05.562375 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a7d6d187-b5c8-44a9-bccf-af998cfad909 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c393b6a3ec3141c3aff392278b481b31', 'request_id': 'req-a7d6d187-b5c8-44a9-bccf-af998cfad909'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:05.577960 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a7d6d187-b5c8-44a9-bccf-af998cfad909 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c393b6a3ec3141c3aff392278b481b31', 'request_id': 'req-a7d6d187-b5c8-44a9-bccf-af998cfad909'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:05.595548 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a7d6d187-b5c8-44a9-bccf-af998cfad909 admin admin] The token cache is being invalidate because project c393b6a3ec3141c3aff392278b481b31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:04:05.595888 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a7d6d187-b5c8-44a9-bccf-af998cfad909 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:05.596785 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a7d6d187-b5c8-44a9-bccf-af998cfad909 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:05.600285 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 988/1960] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:04:05 2022] DELETE /identity/v3/projects/c393b6a3ec3141c3aff392278b481b31 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:04:05.605540 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b332147-a7eb-4145-8a73-21845639f86a None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:05.605907 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b332147-a7eb-4145-8a73-21845639f86a None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:05.606320 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b332147-a7eb-4145-8a73-21845639f86a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:05.623167 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:04:05.636929 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-4b332147-a7eb-4145-8a73-21845639f86a None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:04:05.639204 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-4b332147-a7eb-4145-8a73-21845639f86a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:05.713075 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 973/1961] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:04:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:04:05.722079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c626f-7853-4149-9b5a-587888f13f50 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:05.722751 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c626f-7853-4149-9b5a-587888f13f50 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:05.723033 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c626f-7853-4149-9b5a-587888f13f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:05.749675 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-1c2c626f-7853-4149-9b5a-587888f13f50 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:04:05.751275 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-1c2c626f-7853-4149-9b5a-587888f13f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:05.775031 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 989/1962] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:04:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:04:05.779643 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b332147-a7eb-4145-8a73-21845639f86a None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:05.786793 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c626f-7853-4149-9b5a-587888f13f50 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:05.789529 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-1c2c626f-7853-4149-9b5a-587888f13f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:05.801405 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b332147-a7eb-4145-8a73-21845639f86a None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:05.815277 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c626f-7853-4149-9b5a-587888f13f50 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:05.816355 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad1da1f-9486-47cd-8d47-54b92f7dfe11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:05.817193 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1da1f-9486-47cd-8d47-54b92f7dfe11 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:05.817570 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1da1f-9486-47cd-8d47-54b92f7dfe11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:05.817923 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1da1f-9486-47cd-8d47-54b92f7dfe11 admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:05.819600 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad1da1f-9486-47cd-8d47-54b92f7dfe11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:04:05.820326 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad1da1f-9486-47cd-8d47-54b92f7dfe11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:05.825171 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad1da1f-9486-47cd-8d47-54b92f7dfe11 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:05.834073 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd79f2c-66fe-4207-80c7-619a073c7990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:05.837706 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd79f2c-66fe-4207-80c7-619a073c7990 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:05.838816 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd79f2c-66fe-4207-80c7-619a073c7990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:05.839306 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd79f2c-66fe-4207-80c7-619a073c7990 admin admin] PATH_INFO: `/v3/projects/efaa643308264befaec40cb9ba5d4089` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:05.842916 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd79f2c-66fe-4207-80c7-619a073c7990 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efaa643308264befaec40cb9ba5d4089)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:05.860070 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd79f2c-66fe-4207-80c7-619a073c7990 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:05.866815 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 974/1963] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:04:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:04:05.873260 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad1da1f-9486-47cd-8d47-54b92f7dfe11 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:05.888388 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad1da1f-9486-47cd-8d47-54b92f7dfe11 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:05.913595 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b098af-e8a8-4dc1-beee-7c6717c8100e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:05.914290 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b098af-e8a8-4dc1-beee-7c6717c8100e admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:05.914576 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b098af-e8a8-4dc1-beee-7c6717c8100e admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:05.914858 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b098af-e8a8-4dc1-beee-7c6717c8100e admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:05.915835 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b098af-e8a8-4dc1-beee-7c6717c8100e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:05.918569 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b098af-e8a8-4dc1-beee-7c6717c8100e admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:05.977267 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 975/1964] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:04:05 2022] POST /identity/v3/projects => generated 407 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:04:05.983789 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3cd79f2c-66fe-4207-80c7-619a073c7990 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efaa643308264befaec40cb9ba5d4089', 'request_id': 'req-3cd79f2c-66fe-4207-80c7-619a073c7990'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:05.984366 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b098af-e8a8-4dc1-beee-7c6717c8100e admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:06.000090 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3cd79f2c-66fe-4207-80c7-619a073c7990 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efaa643308264befaec40cb9ba5d4089', 'request_id': 'req-3cd79f2c-66fe-4207-80c7-619a073c7990'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:06.010907 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3cd79f2c-66fe-4207-80c7-619a073c7990 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efaa643308264befaec40cb9ba5d4089', 'request_id': 'req-3cd79f2c-66fe-4207-80c7-619a073c7990'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:06.033791 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3cd79f2c-66fe-4207-80c7-619a073c7990 admin admin] The token cache is being invalidate because project efaa643308264befaec40cb9ba5d4089 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:04:06.034153 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3cd79f2c-66fe-4207-80c7-619a073c7990 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:06.035771 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3cd79f2c-66fe-4207-80c7-619a073c7990 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:06.037972 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 990/1965] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:04:05 2022] DELETE /identity/v3/projects/efaa643308264befaec40cb9ba5d4089 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:04:06.042049 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd79f2c-66fe-4207-80c7-619a073c7990 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:06.044056 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-3cd79f2c-66fe-4207-80c7-619a073c7990 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:06.128896 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b098af-e8a8-4dc1-beee-7c6717c8100e admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:06.134572 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef1a402-64aa-4d24-9fdc-1fd9da530c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:06.140892 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1a402-64aa-4d24-9fdc-1fd9da530c28 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:06.141749 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1a402-64aa-4d24-9fdc-1fd9da530c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:06.142227 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1a402-64aa-4d24-9fdc-1fd9da530c28 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:06.143147 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef1a402-64aa-4d24-9fdc-1fd9da530c28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:06.145861 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef1a402-64aa-4d24-9fdc-1fd9da530c28 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:06.200178 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 976/1966] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:04:05 2022] POST /identity/v3/users => generated 304 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:04:06.204476 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef1a402-64aa-4d24-9fdc-1fd9da530c28 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:06.206885 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-2ef1a402-64aa-4d24-9fdc-1fd9da530c28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:06.222785 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef1a402-64aa-4d24-9fdc-1fd9da530c28 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:06.228403 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6039215-ae30-40da-ba4d-5ee0f502b3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:06.231362 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6039215-ae30-40da-ba4d-5ee0f502b3d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:06.231362 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6039215-ae30-40da-ba4d-5ee0f502b3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:06.231362 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6039215-ae30-40da-ba4d-5ee0f502b3d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:06.233759 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6039215-ae30-40da-ba4d-5ee0f502b3d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:06.233759 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6039215-ae30-40da-ba4d-5ee0f502b3d9 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:06.237711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd79f2c-66fe-4207-80c7-619a073c7990 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:06.247871 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ef98f7-ff0a-4657-9449-b7fd4f97ac73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:06.248960 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef98f7-ff0a-4657-9449-b7fd4f97ac73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:06.249405 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef98f7-ff0a-4657-9449-b7fd4f97ac73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:06.249675 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef98f7-ff0a-4657-9449-b7fd4f97ac73 admin admin] PATH_INFO: `/v3/users/0d119e3f1567445ca4bf46d1c71e1a19` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:06.251443 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ef98f7-ff0a-4657-9449-b7fd4f97ac73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d119e3f1567445ca4bf46d1c71e1a19)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:06.253382 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ef98f7-ff0a-4657-9449-b7fd4f97ac73 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:06.255091 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:04:06.276082 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 977/1967] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 22:04:06.284412 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6039215-ae30-40da-ba4d-5ee0f502b3d9 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:06.351208 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c9ef98f7-ff0a-4657-9449-b7fd4f97ac73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d119e3f1567445ca4bf46d1c71e1a19'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:06.365259 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6039215-ae30-40da-ba4d-5ee0f502b3d9 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:06.372809 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c9ef98f7-ff0a-4657-9449-b7fd4f97ac73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d119e3f1567445ca4bf46d1c71e1a19'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:06.390230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c9ef98f7-ff0a-4657-9449-b7fd4f97ac73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d119e3f1567445ca4bf46d1c71e1a19'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:06.391468 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c9ef98f7-ff0a-4657-9449-b7fd4f97ac73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d119e3f1567445ca4bf46d1c71e1a19'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:06.393213 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c9ef98f7-ff0a-4657-9449-b7fd4f97ac73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d119e3f1567445ca4bf46d1c71e1a19'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:06.396841 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726f9de3-f54d-46bf-87d5-c31aa17ae54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:06.399066 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726f9de3-f54d-46bf-87d5-c31aa17ae54f admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:06.401092 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726f9de3-f54d-46bf-87d5-c31aa17ae54f admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:06.401519 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726f9de3-f54d-46bf-87d5-c31aa17ae54f admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:06.403643 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726f9de3-f54d-46bf-87d5-c31aa17ae54f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:06.407271 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726f9de3-f54d-46bf-87d5-c31aa17ae54f admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:06.426664 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 991/1968] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:04:06 2022] DELETE /identity/v3/users/0d119e3f1567445ca4bf46d1c71e1a19 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:04:06.466362 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 978/1969] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:04:06.470904 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ef98f7-ff0a-4657-9449-b7fd4f97ac73 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:06.477456 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c9ef98f7-ff0a-4657-9449-b7fd4f97ac73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:06.620976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ef98f7-ff0a-4657-9449-b7fd4f97ac73 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:06.638996 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba5b08b-d857-4b45-9add-92548e190d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:06.640114 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba5b08b-d857-4b45-9add-92548e190d97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:06.640410 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba5b08b-d857-4b45-9add-92548e190d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:06.640695 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba5b08b-d857-4b45-9add-92548e190d97 admin admin] PATH_INFO: `/v3/projects/731254cabc2141ffacdd03a76638a54f/users/7b08f789097643949726477099800fa2/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:06.671251 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba5b08b-d857-4b45-9add-92548e190d97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=731254cabc2141ffacdd03a76638a54f, user_id=7b08f789097643949726477099800fa2, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:06.674716 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba5b08b-d857-4b45-9add-92548e190d97 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:06.701271 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 992/1970] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:04:06 2022] PUT /identity/v3/projects/731254cabc2141ffacdd03a76638a54f/users/7b08f789097643949726477099800fa2/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:04:06.714624 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c4ee7-1000-4a01-84bf-dfd001f5187e None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:06.715584 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c4ee7-1000-4a01-84bf-dfd001f5187e None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:06.715584 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c4ee7-1000-4a01-84bf-dfd001f5187e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:06.747293 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-ef7c4ee7-1000-4a01-84bf-dfd001f5187e None None] MFA Rules not processed for user `7b08f789097643949726477099800fa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:04:06.748655 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-ef7c4ee7-1000-4a01-84bf-dfd001f5187e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:06.793594 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:04:06.835983 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 979/1971] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:04:06 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:04:06.841030 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba5b08b-d857-4b45-9add-92548e190d97 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:06.906503 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba5b08b-d857-4b45-9add-92548e190d97 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:06.911240 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da0ed59-88a5-4da2-9fb5-864388aaacba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:06.911942 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da0ed59-88a5-4da2-9fb5-864388aaacba admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:06.912230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da0ed59-88a5-4da2-9fb5-864388aaacba admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:06.912508 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da0ed59-88a5-4da2-9fb5-864388aaacba admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:06.913222 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da0ed59-88a5-4da2-9fb5-864388aaacba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:06.915133 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da0ed59-88a5-4da2-9fb5-864388aaacba admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:06.921673 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:04:06.942047 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c4ee7-1000-4a01-84bf-dfd001f5187e None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:06.944778 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-ef7c4ee7-1000-4a01-84bf-dfd001f5187e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:06.972779 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 993/1972] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:04:06 2022] POST /identity/v3/projects => generated 409 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:04:06.977298 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da0ed59-88a5-4da2-9fb5-864388aaacba admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:06.995654 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c4ee7-1000-4a01-84bf-dfd001f5187e None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:07.062436 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da0ed59-88a5-4da2-9fb5-864388aaacba admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:07.067158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd52b12b-e6e5-4090-8815-2df33eb905be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:07.067931 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd52b12b-e6e5-4090-8815-2df33eb905be admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:07.068212 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd52b12b-e6e5-4090-8815-2df33eb905be admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:07.068485 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd52b12b-e6e5-4090-8815-2df33eb905be admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:07.069202 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd52b12b-e6e5-4090-8815-2df33eb905be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:07.071413 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd52b12b-e6e5-4090-8815-2df33eb905be admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:07.077427 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9418c26-07ef-40f3-bd59-5caea7335c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:07.078462 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9418c26-07ef-40f3-bd59-5caea7335c73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:07.078750 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9418c26-07ef-40f3-bd59-5caea7335c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:07.079048 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9418c26-07ef-40f3-bd59-5caea7335c73 admin admin] PATH_INFO: `/v3/projects/8c9f461e575c4b0b84271fd61593cb62` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:07.080309 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9418c26-07ef-40f3-bd59-5caea7335c73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c9f461e575c4b0b84271fd61593cb62)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:07.082239 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9418c26-07ef-40f3-bd59-5caea7335c73 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:07.096179 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 994/1973] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:04:06 2022] POST /identity/v3/users => generated 311 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:04:07.099954 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd52b12b-e6e5-4090-8815-2df33eb905be admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:07.111170 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd52b12b-e6e5-4090-8815-2df33eb905be admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:07.116508 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f236c2ec-fbe4-44d9-93a4-2c70dbad3a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:07.117230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236c2ec-fbe4-44d9-93a4-2c70dbad3a79 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:07.117516 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236c2ec-fbe4-44d9-93a4-2c70dbad3a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:07.117830 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236c2ec-fbe4-44d9-93a4-2c70dbad3a79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:07.118592 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f236c2ec-fbe4-44d9-93a4-2c70dbad3a79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:07.124777 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f236c2ec-fbe4-44d9-93a4-2c70dbad3a79 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:07.133893 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-e9418c26-07ef-40f3-bd59-5caea7335c73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c9f461e575c4b0b84271fd61593cb62', 'request_id': 'req-e9418c26-07ef-40f3-bd59-5caea7335c73'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:07.136527 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 995/1974] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 22:04:07.141906 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f236c2ec-fbe4-44d9-93a4-2c70dbad3a79 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:07.143606 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-e9418c26-07ef-40f3-bd59-5caea7335c73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c9f461e575c4b0b84271fd61593cb62', 'request_id': 'req-e9418c26-07ef-40f3-bd59-5caea7335c73'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:07.151381 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-e9418c26-07ef-40f3-bd59-5caea7335c73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c9f461e575c4b0b84271fd61593cb62', 'request_id': 'req-e9418c26-07ef-40f3-bd59-5caea7335c73'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:07.152552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f236c2ec-fbe4-44d9-93a4-2c70dbad3a79 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:07.168173 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-e9418c26-07ef-40f3-bd59-5caea7335c73 admin admin] The token cache is being invalidate because project 8c9f461e575c4b0b84271fd61593cb62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:04:07.168476 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-e9418c26-07ef-40f3-bd59-5caea7335c73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:07.169239 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-e9418c26-07ef-40f3-bd59-5caea7335c73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:07.170824 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 980/1975] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:04:06 2022] DELETE /identity/v3/projects/8c9f461e575c4b0b84271fd61593cb62 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:04:07.174715 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9418c26-07ef-40f3-bd59-5caea7335c73 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:07.176088 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-e9418c26-07ef-40f3-bd59-5caea7335c73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:07.221182 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7595eec2-5a0a-4fdc-b673-4dbb8720857d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:07.222127 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595eec2-5a0a-4fdc-b673-4dbb8720857d admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:07.222412 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595eec2-5a0a-4fdc-b673-4dbb8720857d admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:07.222690 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595eec2-5a0a-4fdc-b673-4dbb8720857d admin admin] PATH_INFO: `/v3/projects/293c93b714bb480cb656731f8e126e66/users/8351b8d81b0f40bda9f970f681ab86b2/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:07.240986 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7595eec2-5a0a-4fdc-b673-4dbb8720857d admin admin] RBAC: Authorizing `identity:create_grant(project_id=293c93b714bb480cb656731f8e126e66, user_id=8351b8d81b0f40bda9f970f681ab86b2, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:07.244057 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7595eec2-5a0a-4fdc-b673-4dbb8720857d admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:07.256611 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9418c26-07ef-40f3-bd59-5caea7335c73 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:07.262075 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:04:07.265269 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 996/1976] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:04:07 2022] PUT /identity/v3/projects/293c93b714bb480cb656731f8e126e66/users/8351b8d81b0f40bda9f970f681ab86b2/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:04:07.271310 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7595eec2-5a0a-4fdc-b673-4dbb8720857d admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:07.279922 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-7595eec2-5a0a-4fdc-b673-4dbb8720857d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:07.357734 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:04:07.359721 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6248bf9-0393-4d7a-aafa-4256b93285f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:07.361511 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6248bf9-0393-4d7a-aafa-4256b93285f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:07.361511 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6248bf9-0393-4d7a-aafa-4256b93285f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:07.361797 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6248bf9-0393-4d7a-aafa-4256b93285f5 admin admin] PATH_INFO: `/v3/users/1960724ba28a478d8cc317343f18e8db` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:07.363123 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6248bf9-0393-4d7a-aafa-4256b93285f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1960724ba28a478d8cc317343f18e8db)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:07.365911 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6248bf9-0393-4d7a-aafa-4256b93285f5 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:07.381002 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7595eec2-5a0a-4fdc-b673-4dbb8720857d admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:07.387850 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43ab246-b4d1-433a-8114-5a06868d2ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:07.388759 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ab246-b4d1-433a-8114-5a06868d2ba0 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:07.389194 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ab246-b4d1-433a-8114-5a06868d2ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:07.389632 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ab246-b4d1-433a-8114-5a06868d2ba0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:07.390576 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43ab246-b4d1-433a-8114-5a06868d2ba0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:07.392539 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43ab246-b4d1-433a-8114-5a06868d2ba0 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:07.404489 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 997/1977] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:04:07.409058 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43ab246-b4d1-433a-8114-5a06868d2ba0 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:07.423462 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43ab246-b4d1-433a-8114-5a06868d2ba0 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:07.423711 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d6248bf9-0393-4d7a-aafa-4256b93285f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1960724ba28a478d8cc317343f18e8db'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:07.424215 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d6248bf9-0393-4d7a-aafa-4256b93285f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1960724ba28a478d8cc317343f18e8db'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:07.429532 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def1652-5e20-4d49-9551-7b88200ee3a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:07.430555 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def1652-5e20-4d49-9551-7b88200ee3a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:07.430857 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def1652-5e20-4d49-9551-7b88200ee3a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:07.431163 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def1652-5e20-4d49-9551-7b88200ee3a2 admin admin] PATH_INFO: `/v3/projects/293c93b714bb480cb656731f8e126e66/users/8351b8d81b0f40bda9f970f681ab86b2/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:07.432616 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d6248bf9-0393-4d7a-aafa-4256b93285f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1960724ba28a478d8cc317343f18e8db'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:07.434434 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def1652-5e20-4d49-9551-7b88200ee3a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=293c93b714bb480cb656731f8e126e66, user_id=8351b8d81b0f40bda9f970f681ab86b2, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:07.437254 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def1652-5e20-4d49-9551-7b88200ee3a2 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:07.446325 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d6248bf9-0393-4d7a-aafa-4256b93285f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1960724ba28a478d8cc317343f18e8db'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:07.449728 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 998/1978] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:04:07 2022] PUT /identity/v3/projects/293c93b714bb480cb656731f8e126e66/users/8351b8d81b0f40bda9f970f681ab86b2/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:04:07.454535 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d6248bf9-0393-4d7a-aafa-4256b93285f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1960724ba28a478d8cc317343f18e8db'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:07.460756 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b72c3-064d-4e25-ba76-93e26eac272f None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:07.461052 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b72c3-064d-4e25-ba76-93e26eac272f None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:07.461331 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b72c3-064d-4e25-ba76-93e26eac272f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:07.462183 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 981/1979] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:04:07 2022] DELETE /identity/v3/users/1960724ba28a478d8cc317343f18e8db => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:04:07.496433 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-970b72c3-064d-4e25-ba76-93e26eac272f None None] MFA Rules not processed for user `8351b8d81b0f40bda9f970f681ab86b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:04:07.497774 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-970b72c3-064d-4e25-ba76-93e26eac272f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:07.599912 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 999/1980] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:04:07 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:04:07.611565 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6248bf9-0393-4d7a-aafa-4256b93285f5 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:07.612900 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-d6248bf9-0393-4d7a-aafa-4256b93285f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:07.696041 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6248bf9-0393-4d7a-aafa-4256b93285f5 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:07.703629 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81536197-d1c6-4667-b91d-567e2df8e675 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:07.704570 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81536197-d1c6-4667-b91d-567e2df8e675 service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:07.705173 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81536197-d1c6-4667-b91d-567e2df8e675 service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:07.705518 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81536197-d1c6-4667-b91d-567e2df8e675 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:07.723053 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81536197-d1c6-4667-b91d-567e2df8e675 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:07.725228 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81536197-d1c6-4667-b91d-567e2df8e675 service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:07.742242 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 982/1981] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:04:07 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:04:07.847886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970b72c3-064d-4e25-ba76-93e26eac272f None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:07.849478 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-970b72c3-064d-4e25-ba76-93e26eac272f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:07.973722 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970b72c3-064d-4e25-ba76-93e26eac272f None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:07.983065 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b638f61-524b-427e-8bcc-44be4cdde1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:07.984601 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b638f61-524b-427e-8bcc-44be4cdde1e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:07.984889 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b638f61-524b-427e-8bcc-44be4cdde1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:07.985231 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b638f61-524b-427e-8bcc-44be4cdde1e3 admin admin] PATH_INFO: `/v3/projects/18b7ee947b1e47adbc5cdb79d8ab2439` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:07.986392 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b638f61-524b-427e-8bcc-44be4cdde1e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18b7ee947b1e47adbc5cdb79d8ab2439)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:07.991081 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b638f61-524b-427e-8bcc-44be4cdde1e3 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:07.996459 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81536197-d1c6-4667-b91d-567e2df8e675 service cinder] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:08.006585 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81536197-d1c6-4667-b91d-567e2df8e675 service cinder] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:08.020592 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caecb059-8e4a-4e77-a35f-3e2db8805af2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:08.021274 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caecb059-8e4a-4e77-a35f-3e2db8805af2 service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:08.021553 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caecb059-8e4a-4e77-a35f-3e2db8805af2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:08.021837 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caecb059-8e4a-4e77-a35f-3e2db8805af2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:08.024048 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-caecb059-8e4a-4e77-a35f-3e2db8805af2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:08.050686 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-1b638f61-524b-427e-8bcc-44be4cdde1e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18b7ee947b1e47adbc5cdb79d8ab2439', 'request_id': 'req-1b638f61-524b-427e-8bcc-44be4cdde1e3'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:08.063889 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:04:08.072160 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-1b638f61-524b-427e-8bcc-44be4cdde1e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18b7ee947b1e47adbc5cdb79d8ab2439', 'request_id': 'req-1b638f61-524b-427e-8bcc-44be4cdde1e3'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:08.075303 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:04:08.084697 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-1b638f61-524b-427e-8bcc-44be4cdde1e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18b7ee947b1e47adbc5cdb79d8ab2439', 'request_id': 'req-1b638f61-524b-427e-8bcc-44be4cdde1e3'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:08.099313 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-1b638f61-524b-427e-8bcc-44be4cdde1e3 admin admin] The token cache is being invalidate because project 18b7ee947b1e47adbc5cdb79d8ab2439 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:04:08.099738 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-1b638f61-524b-427e-8bcc-44be4cdde1e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:08.100528 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-1b638f61-524b-427e-8bcc-44be4cdde1e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:08.103505 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1000/1982] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:04:07 2022] DELETE /identity/v3/projects/18b7ee947b1e47adbc5cdb79d8ab2439 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:04:08.107690 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caecb059-8e4a-4e77-a35f-3e2db8805af2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:08.108440 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe7bcf-f16e-4d3a-bc28-ae47d20feb3d None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:08.108733 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe7bcf-f16e-4d3a-bc28-ae47d20feb3d None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:08.109016 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe7bcf-f16e-4d3a-bc28-ae47d20feb3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:08.110227 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caecb059-8e4a-4e77-a35f-3e2db8805af2 service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:08.119955 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 983/1983] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:04:07 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:04:08.137365 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-f1fe7bcf-f16e-4d3a-bc28-ae47d20feb3d None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:04:08.138550 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-f1fe7bcf-f16e-4d3a-bc28-ae47d20feb3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:08.227692 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1001/1984] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:04:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:04:08.233032 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bd8c75-f1e5-4235-9d76-e4f873ac7c9e None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:08.233328 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bd8c75-f1e5-4235-9d76-e4f873ac7c9e None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:08.233619 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bd8c75-f1e5-4235-9d76-e4f873ac7c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:08.254764 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-14bd8c75-f1e5-4235-9d76-e4f873ac7c9e None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:04:08.256807 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-14bd8c75-f1e5-4235-9d76-e4f873ac7c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:08.270840 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 984/1985] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:04:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:04:08.272819 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fe7bcf-f16e-4d3a-bc28-ae47d20feb3d None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:08.288233 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fe7bcf-f16e-4d3a-bc28-ae47d20feb3d None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:08.294725 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ed6bd7-0533-452f-b7e5-8c0ee7cc8680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:08.296062 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed6bd7-0533-452f-b7e5-8c0ee7cc8680 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:08.296759 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed6bd7-0533-452f-b7e5-8c0ee7cc8680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:08.297224 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed6bd7-0533-452f-b7e5-8c0ee7cc8680 admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:08.298618 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ed6bd7-0533-452f-b7e5-8c0ee7cc8680 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:04:08.298982 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ed6bd7-0533-452f-b7e5-8c0ee7cc8680 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:08.301320 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ed6bd7-0533-452f-b7e5-8c0ee7cc8680 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:08.317550 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1002/1986] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:04:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 22:04:08.322004 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bd8c75-f1e5-4235-9d76-e4f873ac7c9e None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:08.334798 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bd8c75-f1e5-4235-9d76-e4f873ac7c9e None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:08.346617 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0a57a7-48a7-4348-b8df-05a269e653db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:08.347433 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a57a7-48a7-4348-b8df-05a269e653db admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:08.347726 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a57a7-48a7-4348-b8df-05a269e653db admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:08.348071 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a57a7-48a7-4348-b8df-05a269e653db admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:08.349106 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0a57a7-48a7-4348-b8df-05a269e653db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:08.353681 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0a57a7-48a7-4348-b8df-05a269e653db admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:08.399245 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 985/1987] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:04:08 2022] POST /identity/v3/projects => generated 383 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:04:08.400201 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ed6bd7-0533-452f-b7e5-8c0ee7cc8680 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:08.412304 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ed6bd7-0533-452f-b7e5-8c0ee7cc8680 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:08.582105 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa21a2-3b7a-4627-ac78-fdc9feff20f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:08.583152 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa21a2-3b7a-4627-ac78-fdc9feff20f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:08.583922 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa21a2-3b7a-4627-ac78-fdc9feff20f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:08.584417 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa21a2-3b7a-4627-ac78-fdc9feff20f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:08.585367 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aa21a2-3b7a-4627-ac78-fdc9feff20f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:08.587597 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aa21a2-3b7a-4627-ac78-fdc9feff20f4 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:08.625303 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1003/1988] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:04:08 2022] POST /identity/v3/users => generated 292 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:04:08.630232 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0a57a7-48a7-4348-b8df-05a269e653db admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:08.654019 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0a57a7-48a7-4348-b8df-05a269e653db admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:08.661412 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1350e4c-87de-4401-895d-0b22bb1f25e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:08.663795 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1350e4c-87de-4401-895d-0b22bb1f25e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:08.664550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1350e4c-87de-4401-895d-0b22bb1f25e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:08.665007 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1350e4c-87de-4401-895d-0b22bb1f25e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:08.666402 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1350e4c-87de-4401-895d-0b22bb1f25e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:08.668892 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1350e4c-87de-4401-895d-0b22bb1f25e1 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:08.682844 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 986/1989] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:04:08.687700 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa21a2-3b7a-4627-ac78-fdc9feff20f4 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:08.698640 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa21a2-3b7a-4627-ac78-fdc9feff20f4 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:08.703370 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e422f-675d-4276-ab50-d955bdd1f3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:08.704074 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e422f-675d-4276-ab50-d955bdd1f3ca admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:08.704358 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e422f-675d-4276-ab50-d955bdd1f3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:08.704634 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e422f-675d-4276-ab50-d955bdd1f3ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:08.705367 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7e422f-675d-4276-ab50-d955bdd1f3ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:08.706957 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7e422f-675d-4276-ab50-d955bdd1f3ca admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:08.724565 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1004/1990] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:04:08.728643 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1350e4c-87de-4401-895d-0b22bb1f25e1 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:08.742796 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1350e4c-87de-4401-895d-0b22bb1f25e1 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:08.756080 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060b73db-8f1d-46fb-bf86-3d9d9b0de208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:08.757343 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b73db-8f1d-46fb-bf86-3d9d9b0de208 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:08.757824 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b73db-8f1d-46fb-bf86-3d9d9b0de208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:08.758336 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b73db-8f1d-46fb-bf86-3d9d9b0de208 admin admin] PATH_INFO: `/v3/projects/d3b84dd581fc4e9499cfecbb7242e3b3/users/fa4fc7744b3d4e5fb6e281fcc90cef5e/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:08.789597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060b73db-8f1d-46fb-bf86-3d9d9b0de208 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3b84dd581fc4e9499cfecbb7242e3b3, user_id=fa4fc7744b3d4e5fb6e281fcc90cef5e, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:08.793669 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060b73db-8f1d-46fb-bf86-3d9d9b0de208 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:08.814468 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 987/1991] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:04:08 2022] PUT /identity/v3/projects/d3b84dd581fc4e9499cfecbb7242e3b3/users/fa4fc7744b3d4e5fb6e281fcc90cef5e/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:04:08.819533 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b54cb3-f9fe-4fd5-807b-90d50b86b9a8 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:08.819858 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b54cb3-f9fe-4fd5-807b-90d50b86b9a8 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:08.820144 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b54cb3-f9fe-4fd5-807b-90d50b86b9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:08.840515 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-a7b54cb3-f9fe-4fd5-807b-90d50b86b9a8 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:04:08.841829 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a7b54cb3-f9fe-4fd5-807b-90d50b86b9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:08.890588 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:04:08.907150 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1005/1992] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:04:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:04:08.916941 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060b73db-8f1d-46fb-bf86-3d9d9b0de208 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:08.918909 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-060b73db-8f1d-46fb-bf86-3d9d9b0de208 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:08.984663 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060b73db-8f1d-46fb-bf86-3d9d9b0de208 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:08.988318 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5929ffa5-e081-468e-9f37-39e19270cd22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:08.988970 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929ffa5-e081-468e-9f37-39e19270cd22 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:08.989257 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929ffa5-e081-468e-9f37-39e19270cd22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:08.989540 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929ffa5-e081-468e-9f37-39e19270cd22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:08.998722 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5929ffa5-e081-468e-9f37-39e19270cd22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:09.000179 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5929ffa5-e081-468e-9f37-39e19270cd22 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:09.010313 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 988/1993] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:04:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:04:09.810802 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fb90f-2202-4caf-8510-7ce7d7356dde None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:09.811182 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fb90f-2202-4caf-8510-7ce7d7356dde None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:09.811403 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fb90f-2202-4caf-8510-7ce7d7356dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:09.897463 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-616fb90f-2202-4caf-8510-7ce7d7356dde None None] MFA Rules not processed for user `fa4fc7744b3d4e5fb6e281fcc90cef5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:04:09.898754 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-616fb90f-2202-4caf-8510-7ce7d7356dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:10.007840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1006/1994] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:04:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:04:10.017861 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5929ffa5-e081-468e-9f37-39e19270cd22 service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:10.019395 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-5929ffa5-e081-468e-9f37-39e19270cd22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:10.098503 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5929ffa5-e081-468e-9f37-39e19270cd22 service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:10.103406 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df0cd0-b2a1-4372-8658-ac73c9b707e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:10.105344 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df0cd0-b2a1-4372-8658-ac73c9b707e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:10.105893 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df0cd0-b2a1-4372-8658-ac73c9b707e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:10.106217 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df0cd0-b2a1-4372-8658-ac73c9b707e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:10.116913 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80df0cd0-b2a1-4372-8658-ac73c9b707e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:10.118638 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80df0cd0-b2a1-4372-8658-ac73c9b707e6 service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:10.131304 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 989/1995] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:04:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:04:10.552539 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684a979-f02e-417e-a8ff-38680d5f3d04 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:10.552922 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684a979-f02e-417e-a8ff-38680d5f3d04 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:10.553223 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684a979-f02e-417e-a8ff-38680d5f3d04 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:10.554602 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1007/1996] 10.222.0.13 () {54 vars in 921 bytes} [Tue Jun 7 22:04:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 22:04:10.591487 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432de3f4-409a-43cf-83ee-a138ccfc33d1 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:10.591826 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432de3f4-409a-43cf-83ee-a138ccfc33d1 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:10.592109 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432de3f4-409a-43cf-83ee-a138ccfc33d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:10.612701 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-432de3f4-409a-43cf-83ee-a138ccfc33d1 None None] MFA Rules not processed for user `02941c6ad2d54e48b1114d9e030c66c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:04:10.614198 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-432de3f4-409a-43cf-83ee-a138ccfc33d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:10.619020 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:04:10.683721 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 990/1997] 10.222.0.13 () {58 vars in 1018 bytes} [Tue Jun 7 22:04:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:04:10.694906 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7684a979-f02e-417e-a8ff-38680d5f3d04 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:10.696581 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-7684a979-f02e-417e-a8ff-38680d5f3d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:10.716060 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7684a979-f02e-417e-a8ff-38680d5f3d04 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:10.722914 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea98e93-3aac-4b4e-b503-77763be627b3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02941c6ad2d54e48b1114d9e030c66c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:10.724361 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea98e93-3aac-4b4e-b503-77763be627b3 service placement] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:10.724645 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea98e93-3aac-4b4e-b503-77763be627b3 service placement] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:10.725359 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea98e93-3aac-4b4e-b503-77763be627b3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:10.734935 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea98e93-3aac-4b4e-b503-77763be627b3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:10.736896 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea98e93-3aac-4b4e-b503-77763be627b3 service placement] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:10.748718 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1008/1998] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:04:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:04:11.241730 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432de3f4-409a-43cf-83ee-a138ccfc33d1 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:11.243244 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-432de3f4-409a-43cf-83ee-a138ccfc33d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:11.263885 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432de3f4-409a-43cf-83ee-a138ccfc33d1 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:11.269011 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a4bbf5-ef90-4807-8c2f-71dc6370456f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:11.269895 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4bbf5-ef90-4807-8c2f-71dc6370456f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:11.270364 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4bbf5-ef90-4807-8c2f-71dc6370456f admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:11.270642 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4bbf5-ef90-4807-8c2f-71dc6370456f admin admin] PATH_INFO: `/v3/users/2eba784f64284eb1b1621c432e29ae3f` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:11.271883 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a4bbf5-ef90-4807-8c2f-71dc6370456f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2eba784f64284eb1b1621c432e29ae3f)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:11.273801 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a4bbf5-ef90-4807-8c2f-71dc6370456f admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:11.327547 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-a8a4bbf5-ef90-4807-8c2f-71dc6370456f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2eba784f64284eb1b1621c432e29ae3f'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:11.328222 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-a8a4bbf5-ef90-4807-8c2f-71dc6370456f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2eba784f64284eb1b1621c432e29ae3f'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:11.336287 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-a8a4bbf5-ef90-4807-8c2f-71dc6370456f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2eba784f64284eb1b1621c432e29ae3f'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:11.348724 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-a8a4bbf5-ef90-4807-8c2f-71dc6370456f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2eba784f64284eb1b1621c432e29ae3f'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:11.356350 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-a8a4bbf5-ef90-4807-8c2f-71dc6370456f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2eba784f64284eb1b1621c432e29ae3f'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:11.362574 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 991/1999] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:04:11 2022] DELETE /identity/v3/users/2eba784f64284eb1b1621c432e29ae3f => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:04:11.545371 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea98e93-3aac-4b4e-b503-77763be627b3 service placement] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:11.565952 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea98e93-3aac-4b4e-b503-77763be627b3 service placement] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:11.634166 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d5a973-8074-49aa-9df5-9df5218b7a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:11.635356 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5a973-8074-49aa-9df5-9df5218b7a6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:11.635824 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5a973-8074-49aa-9df5-9df5218b7a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:11.636106 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5a973-8074-49aa-9df5-9df5218b7a6b admin admin] PATH_INFO: `/v3/projects/e224d637e96d4c168ac90b79697bb89b` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:11.637303 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d5a973-8074-49aa-9df5-9df5218b7a6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e224d637e96d4c168ac90b79697bb89b)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:11.639379 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d5a973-8074-49aa-9df5-9df5218b7a6b admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:11.685584 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a2d5a973-8074-49aa-9df5-9df5218b7a6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e224d637e96d4c168ac90b79697bb89b', 'request_id': 'req-a2d5a973-8074-49aa-9df5-9df5218b7a6b'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:11.691952 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a2d5a973-8074-49aa-9df5-9df5218b7a6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e224d637e96d4c168ac90b79697bb89b', 'request_id': 'req-a2d5a973-8074-49aa-9df5-9df5218b7a6b'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:11.697611 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a2d5a973-8074-49aa-9df5-9df5218b7a6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e224d637e96d4c168ac90b79697bb89b', 'request_id': 'req-a2d5a973-8074-49aa-9df5-9df5218b7a6b'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:11.705051 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:04:11.709192 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a2d5a973-8074-49aa-9df5-9df5218b7a6b admin admin] The token cache is being invalidate because project e224d637e96d4c168ac90b79697bb89b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:04:11.709937 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a2d5a973-8074-49aa-9df5-9df5218b7a6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:11.710655 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a2d5a973-8074-49aa-9df5-9df5218b7a6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:11.712276 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1009/2000] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:04:11 2022] DELETE /identity/v3/projects/e224d637e96d4c168ac90b79697bb89b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:04:11.717498 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a4bbf5-ef90-4807-8c2f-71dc6370456f admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:11.718984 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-a8a4bbf5-ef90-4807-8c2f-71dc6370456f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:11.733037 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a4bbf5-ef90-4807-8c2f-71dc6370456f admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:11.789619 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed36d76-ff86-4ed3-ace0-e88dc7fc9eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:11.790488 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed36d76-ff86-4ed3-ace0-e88dc7fc9eab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:11.790772 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed36d76-ff86-4ed3-ace0-e88dc7fc9eab admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:11.791078 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed36d76-ff86-4ed3-ace0-e88dc7fc9eab admin admin] PATH_INFO: `/v3/users/18b1f026a28e42dd81a741d024f1d4fb` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:11.792629 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed36d76-ff86-4ed3-ace0-e88dc7fc9eab admin admin] RBAC: Authorizing `identity:delete_user(user_id=18b1f026a28e42dd81a741d024f1d4fb)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:11.794526 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed36d76-ff86-4ed3-ace0-e88dc7fc9eab admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:11.814684 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:04:11.843124 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-3ed36d76-ff86-4ed3-ace0-e88dc7fc9eab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18b1f026a28e42dd81a741d024f1d4fb'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:11.845131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-3ed36d76-ff86-4ed3-ace0-e88dc7fc9eab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18b1f026a28e42dd81a741d024f1d4fb'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:11.852940 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-3ed36d76-ff86-4ed3-ace0-e88dc7fc9eab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18b1f026a28e42dd81a741d024f1d4fb'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:11.868276 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-3ed36d76-ff86-4ed3-ace0-e88dc7fc9eab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18b1f026a28e42dd81a741d024f1d4fb'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:11.875472 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-3ed36d76-ff86-4ed3-ace0-e88dc7fc9eab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18b1f026a28e42dd81a741d024f1d4fb'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:11.889769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 992/2001] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:04:11 2022] DELETE /identity/v3/users/18b1f026a28e42dd81a741d024f1d4fb => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:04:12.061170 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d5a973-8074-49aa-9df5-9df5218b7a6b admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:12.062802 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a2d5a973-8074-49aa-9df5-9df5218b7a6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:12.076503 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d5a973-8074-49aa-9df5-9df5218b7a6b admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:12.134217 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09531fe2-cf4e-4615-a5c3-50163642c226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:12.135792 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09531fe2-cf4e-4615-a5c3-50163642c226 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:12.136119 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09531fe2-cf4e-4615-a5c3-50163642c226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:12.136406 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09531fe2-cf4e-4615-a5c3-50163642c226 admin admin] PATH_INFO: `/v3/projects/0fd0d297527b4701bb31615d46055539` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:12.137577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09531fe2-cf4e-4615-a5c3-50163642c226 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fd0d297527b4701bb31615d46055539)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:12.139529 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09531fe2-cf4e-4615-a5c3-50163642c226 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:12.187081 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-09531fe2-cf4e-4615-a5c3-50163642c226 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fd0d297527b4701bb31615d46055539', 'request_id': 'req-09531fe2-cf4e-4615-a5c3-50163642c226'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:12.194812 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-09531fe2-cf4e-4615-a5c3-50163642c226 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fd0d297527b4701bb31615d46055539', 'request_id': 'req-09531fe2-cf4e-4615-a5c3-50163642c226'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:12.201395 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-09531fe2-cf4e-4615-a5c3-50163642c226 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fd0d297527b4701bb31615d46055539', 'request_id': 'req-09531fe2-cf4e-4615-a5c3-50163642c226'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:12.212326 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-09531fe2-cf4e-4615-a5c3-50163642c226 admin admin] The token cache is being invalidate because project 0fd0d297527b4701bb31615d46055539 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:04:12.212612 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-09531fe2-cf4e-4615-a5c3-50163642c226 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:12.213319 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-09531fe2-cf4e-4615-a5c3-50163642c226 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:12.215034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1010/2002] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:04:12 2022] DELETE /identity/v3/projects/0fd0d297527b4701bb31615d46055539 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:04:12.219989 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b2747-5ad2-4065-8a08-c27efc6a57f3 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:12.220286 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b2747-5ad2-4065-8a08-c27efc6a57f3 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:12.220568 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b2747-5ad2-4065-8a08-c27efc6a57f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:12.239821 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-5e5b2747-5ad2-4065-8a08-c27efc6a57f3 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:04:12.241046 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-5e5b2747-5ad2-4065-8a08-c27efc6a57f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:12.306054 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 993/2003] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:04:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:04:12.314713 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d717bbb2-6d13-42ac-9032-9c1dd14a2075 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:12.315004 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d717bbb2-6d13-42ac-9032-9c1dd14a2075 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:12.315282 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d717bbb2-6d13-42ac-9032-9c1dd14a2075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:12.343829 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-d717bbb2-6d13-42ac-9032-9c1dd14a2075 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:04:12.345087 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d717bbb2-6d13-42ac-9032-9c1dd14a2075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:12.354125 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1011/2004] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:04:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:04:12.358230 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5b2747-5ad2-4065-8a08-c27efc6a57f3 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:12.368514 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5b2747-5ad2-4065-8a08-c27efc6a57f3 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:12.373906 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891841e5-ae2d-42ff-8746-5d916456db32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:12.374559 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891841e5-ae2d-42ff-8746-5d916456db32 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:12.374839 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891841e5-ae2d-42ff-8746-5d916456db32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:12.375115 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891841e5-ae2d-42ff-8746-5d916456db32 admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:12.375879 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891841e5-ae2d-42ff-8746-5d916456db32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:04:12.376182 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891841e5-ae2d-42ff-8746-5d916456db32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:12.378083 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891841e5-ae2d-42ff-8746-5d916456db32 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:12.390552 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 994/2005] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:04:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:04:12.394702 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d717bbb2-6d13-42ac-9032-9c1dd14a2075 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:12.404614 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d717bbb2-6d13-42ac-9032-9c1dd14a2075 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:12.409624 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b2edcd-c73a-4a53-997f-7b509462182c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:12.410419 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b2edcd-c73a-4a53-997f-7b509462182c admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:12.410767 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b2edcd-c73a-4a53-997f-7b509462182c admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:12.411106 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b2edcd-c73a-4a53-997f-7b509462182c admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:12.411977 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b2edcd-c73a-4a53-997f-7b509462182c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:12.413957 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b2edcd-c73a-4a53-997f-7b509462182c admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:12.443441 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1012/2006] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:04:12 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:04:12.446412 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891841e5-ae2d-42ff-8746-5d916456db32 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:12.457101 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891841e5-ae2d-42ff-8746-5d916456db32 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:12.515048 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec359ebc-c7d6-4737-ac2d-b64d81226d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:12.515750 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec359ebc-c7d6-4737-ac2d-b64d81226d17 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:12.516032 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec359ebc-c7d6-4737-ac2d-b64d81226d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:12.516306 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec359ebc-c7d6-4737-ac2d-b64d81226d17 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:12.516988 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec359ebc-c7d6-4737-ac2d-b64d81226d17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:12.518880 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec359ebc-c7d6-4737-ac2d-b64d81226d17 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:12.545754 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 995/2007] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:04:12 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:04:12.547031 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b2edcd-c73a-4a53-997f-7b509462182c admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:12.558493 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b2edcd-c73a-4a53-997f-7b509462182c admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:12.563409 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9f9c24-bfee-4f92-a9d3-fdabf134b13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:12.564143 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f9c24-bfee-4f92-a9d3-fdabf134b13a admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:12.564543 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f9c24-bfee-4f92-a9d3-fdabf134b13a admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:12.564826 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f9c24-bfee-4f92-a9d3-fdabf134b13a admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:12.565559 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9f9c24-bfee-4f92-a9d3-fdabf134b13a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:12.567177 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9f9c24-bfee-4f92-a9d3-fdabf134b13a admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:12.578077 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1013/2008] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:04:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:04:12.582036 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec359ebc-c7d6-4737-ac2d-b64d81226d17 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:12.592310 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec359ebc-c7d6-4737-ac2d-b64d81226d17 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:12.596844 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d117e-1107-4f11-b686-a82adc0a0294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:12.597530 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d117e-1107-4f11-b686-a82adc0a0294 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:12.597815 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d117e-1107-4f11-b686-a82adc0a0294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:12.598091 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d117e-1107-4f11-b686-a82adc0a0294 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:12.598841 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d117e-1107-4f11-b686-a82adc0a0294 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:12.600445 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d117e-1107-4f11-b686-a82adc0a0294 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:12.611615 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 996/2009] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:04:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:04:12.615620 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9f9c24-bfee-4f92-a9d3-fdabf134b13a admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:12.626764 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9f9c24-bfee-4f92-a9d3-fdabf134b13a admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:12.631520 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b0fab4-1f22-4762-8d66-b8af00e164c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:12.632413 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0fab4-1f22-4762-8d66-b8af00e164c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:12.632698 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0fab4-1f22-4762-8d66-b8af00e164c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:12.632981 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0fab4-1f22-4762-8d66-b8af00e164c3 admin admin] PATH_INFO: `/v3/projects/bddab25e796243688fff05c9b2d4af31/users/661659f9e2b548ac9fd4fd2a3ca58ad5/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:12.647726 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b0fab4-1f22-4762-8d66-b8af00e164c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bddab25e796243688fff05c9b2d4af31, user_id=661659f9e2b548ac9fd4fd2a3ca58ad5, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:12.650443 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b0fab4-1f22-4762-8d66-b8af00e164c3 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:12.663438 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1014/2010] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:04:12 2022] PUT /identity/v3/projects/bddab25e796243688fff05c9b2d4af31/users/661659f9e2b548ac9fd4fd2a3ca58ad5/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:04:12.675875 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da85a84a-7a8e-4122-af24-7cacbd4b5120 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:12.676267 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da85a84a-7a8e-4122-af24-7cacbd4b5120 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:12.676558 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da85a84a-7a8e-4122-af24-7cacbd4b5120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:12.683124 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:04:12.711604 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-da85a84a-7a8e-4122-af24-7cacbd4b5120 None None] MFA Rules not processed for user `661659f9e2b548ac9fd4fd2a3ca58ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:04:12.713426 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-da85a84a-7a8e-4122-af24-7cacbd4b5120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:12.785930 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 997/2011] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:04:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:04:12.798635 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b0fab4-1f22-4762-8d66-b8af00e164c3 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:12.800343 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-95b0fab4-1f22-4762-8d66-b8af00e164c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:12.832646 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:04:12.860933 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b0fab4-1f22-4762-8d66-b8af00e164c3 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:12.864563 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb50f4d-c918-4376-8568-e31d90dfe818 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:12.865211 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb50f4d-c918-4376-8568-e31d90dfe818 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:12.865494 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb50f4d-c918-4376-8568-e31d90dfe818 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:12.865771 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb50f4d-c918-4376-8568-e31d90dfe818 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:12.874235 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb50f4d-c918-4376-8568-e31d90dfe818 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:12.875778 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb50f4d-c918-4376-8568-e31d90dfe818 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:12.884428 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1015/2012] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:04:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:04:16.781954 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de05472-c606-4286-9351-690725b93e3c None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:16.782492 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de05472-c606-4286-9351-690725b93e3c None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:16.782593 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de05472-c606-4286-9351-690725b93e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:16.801326 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-7de05472-c606-4286-9351-690725b93e3c None None] MFA Rules not processed for user `fa4fc7744b3d4e5fb6e281fcc90cef5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:04:16.802551 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-7de05472-c606-4286-9351-690725b93e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:16.870702 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 998/2013] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:04:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:04:22.880449 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=14 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:04:25.330894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb50f4d-c918-4376-8568-e31d90dfe818 service cinder] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:25.332633 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-ceb50f4d-c918-4376-8568-e31d90dfe818 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:25.408380 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb50f4d-c918-4376-8568-e31d90dfe818 service cinder] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:25.413067 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d45a0c-9d0b-4003-b4fe-8dfd2ed5ea6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:25.413956 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d45a0c-9d0b-4003-b4fe-8dfd2ed5ea6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:25.414241 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d45a0c-9d0b-4003-b4fe-8dfd2ed5ea6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:25.414525 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d45a0c-9d0b-4003-b4fe-8dfd2ed5ea6e admin admin] PATH_INFO: `/v3/users/7b08f789097643949726477099800fa2` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:25.416249 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d45a0c-9d0b-4003-b4fe-8dfd2ed5ea6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b08f789097643949726477099800fa2)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:25.418169 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d45a0c-9d0b-4003-b4fe-8dfd2ed5ea6e admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:25.467734 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-b5d45a0c-9d0b-4003-b4fe-8dfd2ed5ea6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b08f789097643949726477099800fa2'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:25.474729 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-b5d45a0c-9d0b-4003-b4fe-8dfd2ed5ea6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b08f789097643949726477099800fa2'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:25.487369 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-b5d45a0c-9d0b-4003-b4fe-8dfd2ed5ea6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b08f789097643949726477099800fa2'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:25.488035 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-b5d45a0c-9d0b-4003-b4fe-8dfd2ed5ea6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b08f789097643949726477099800fa2'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:25.488635 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-b5d45a0c-9d0b-4003-b4fe-8dfd2ed5ea6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b08f789097643949726477099800fa2'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:25.502296 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1016/2014] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:04:25 2022] DELETE /identity/v3/users/7b08f789097643949726477099800fa2 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:04:25.506906 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263e5c8-8a51-464a-a585-5784bf66964e None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:25.507369 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263e5c8-8a51-464a-a585-5784bf66964e None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:25.507505 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263e5c8-8a51-464a-a585-5784bf66964e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:25.524570 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-0263e5c8-8a51-464a-a585-5784bf66964e None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:04:25.525925 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-0263e5c8-8a51-464a-a585-5784bf66964e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:25.592212 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 999/2015] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:04:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:04:25.602003 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d45a0c-9d0b-4003-b4fe-8dfd2ed5ea6e admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:25.604385 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-b5d45a0c-9d0b-4003-b4fe-8dfd2ed5ea6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:25.676668 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d45a0c-9d0b-4003-b4fe-8dfd2ed5ea6e admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:25.680328 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe711245-9a27-47cc-a79d-00acff3ee3cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:25.681002 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe711245-9a27-47cc-a79d-00acff3ee3cd service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:25.681442 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe711245-9a27-47cc-a79d-00acff3ee3cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:25.681855 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe711245-9a27-47cc-a79d-00acff3ee3cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:25.690934 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe711245-9a27-47cc-a79d-00acff3ee3cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:25.692391 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe711245-9a27-47cc-a79d-00acff3ee3cd service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:25.701645 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1017/2016] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:04:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:04:25.976376 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263e5c8-8a51-464a-a585-5784bf66964e None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:25.978078 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-0263e5c8-8a51-464a-a585-5784bf66964e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:25.994437 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263e5c8-8a51-464a-a585-5784bf66964e None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:26.001681 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f847d3-ecc8-4ce0-9f74-cd7f422d379e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:26.002684 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f847d3-ecc8-4ce0-9f74-cd7f422d379e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:26.003054 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f847d3-ecc8-4ce0-9f74-cd7f422d379e admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:26.003535 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f847d3-ecc8-4ce0-9f74-cd7f422d379e admin admin] PATH_INFO: `/v3/projects/731254cabc2141ffacdd03a76638a54f` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:26.004915 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f847d3-ecc8-4ce0-9f74-cd7f422d379e admin admin] RBAC: Authorizing `identity:delete_project(project_id=731254cabc2141ffacdd03a76638a54f)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:26.007284 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f847d3-ecc8-4ce0-9f74-cd7f422d379e admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:26.054906 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:04:26.078482 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-e1f847d3-ecc8-4ce0-9f74-cd7f422d379e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '731254cabc2141ffacdd03a76638a54f', 'request_id': 'req-e1f847d3-ecc8-4ce0-9f74-cd7f422d379e'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:26.085716 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-e1f847d3-ecc8-4ce0-9f74-cd7f422d379e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '731254cabc2141ffacdd03a76638a54f', 'request_id': 'req-e1f847d3-ecc8-4ce0-9f74-cd7f422d379e'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:26.092631 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-e1f847d3-ecc8-4ce0-9f74-cd7f422d379e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '731254cabc2141ffacdd03a76638a54f', 'request_id': 'req-e1f847d3-ecc8-4ce0-9f74-cd7f422d379e'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:26.103115 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-e1f847d3-ecc8-4ce0-9f74-cd7f422d379e admin admin] The token cache is being invalidate because project 731254cabc2141ffacdd03a76638a54f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:04:26.103530 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-e1f847d3-ecc8-4ce0-9f74-cd7f422d379e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:26.104171 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-e1f847d3-ecc8-4ce0-9f74-cd7f422d379e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:26.105777 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1000/2017] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:04:25 2022] DELETE /identity/v3/projects/731254cabc2141ffacdd03a76638a54f => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:04:26.109861 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe711245-9a27-47cc-a79d-00acff3ee3cd service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:26.119406 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe711245-9a27-47cc-a79d-00acff3ee3cd service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:26.137672 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:04:26.180450 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6893854-b619-4481-972b-64346969276f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:26.181959 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6893854-b619-4481-972b-64346969276f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:26.182428 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6893854-b619-4481-972b-64346969276f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:26.182884 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6893854-b619-4481-972b-64346969276f admin admin] PATH_INFO: `/v3/users/8351b8d81b0f40bda9f970f681ab86b2` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:26.184600 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6893854-b619-4481-972b-64346969276f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8351b8d81b0f40bda9f970f681ab86b2)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:26.186565 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6893854-b619-4481-972b-64346969276f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:26.237400 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-f6893854-b619-4481-972b-64346969276f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8351b8d81b0f40bda9f970f681ab86b2'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:26.244448 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-f6893854-b619-4481-972b-64346969276f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8351b8d81b0f40bda9f970f681ab86b2'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:26.258225 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-f6893854-b619-4481-972b-64346969276f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8351b8d81b0f40bda9f970f681ab86b2'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:26.258909 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-f6893854-b619-4481-972b-64346969276f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8351b8d81b0f40bda9f970f681ab86b2'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:26.260372 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-f6893854-b619-4481-972b-64346969276f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8351b8d81b0f40bda9f970f681ab86b2'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:26.273482 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1018/2018] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:04:26 2022] DELETE /identity/v3/users/8351b8d81b0f40bda9f970f681ab86b2 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:04:26.584384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f847d3-ecc8-4ce0-9f74-cd7f422d379e admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:26.585939 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-e1f847d3-ecc8-4ce0-9f74-cd7f422d379e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:26.607834 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f847d3-ecc8-4ce0-9f74-cd7f422d379e admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:26.668039 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c637fb0-228e-43d4-9140-1dbbebe06806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:26.668939 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c637fb0-228e-43d4-9140-1dbbebe06806 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:26.669225 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c637fb0-228e-43d4-9140-1dbbebe06806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:26.669505 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c637fb0-228e-43d4-9140-1dbbebe06806 admin admin] PATH_INFO: `/v3/projects/293c93b714bb480cb656731f8e126e66` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:26.670711 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c637fb0-228e-43d4-9140-1dbbebe06806 admin admin] RBAC: Authorizing `identity:delete_project(project_id=293c93b714bb480cb656731f8e126e66)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:26.672697 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c637fb0-228e-43d4-9140-1dbbebe06806 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:26.744249 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-2c637fb0-228e-43d4-9140-1dbbebe06806 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '293c93b714bb480cb656731f8e126e66', 'request_id': 'req-2c637fb0-228e-43d4-9140-1dbbebe06806'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:26.751275 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-2c637fb0-228e-43d4-9140-1dbbebe06806 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '293c93b714bb480cb656731f8e126e66', 'request_id': 'req-2c637fb0-228e-43d4-9140-1dbbebe06806'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:26.761012 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-2c637fb0-228e-43d4-9140-1dbbebe06806 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '293c93b714bb480cb656731f8e126e66', 'request_id': 'req-2c637fb0-228e-43d4-9140-1dbbebe06806'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:26.772646 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-2c637fb0-228e-43d4-9140-1dbbebe06806 admin admin] The token cache is being invalidate because project 293c93b714bb480cb656731f8e126e66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:04:26.772935 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-2c637fb0-228e-43d4-9140-1dbbebe06806 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:26.773588 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-2c637fb0-228e-43d4-9140-1dbbebe06806 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:26.775173 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1001/2019] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:04:26 2022] DELETE /identity/v3/projects/293c93b714bb480cb656731f8e126e66 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:04:26.780153 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c891f5b-9333-454c-8e76-ca9c0cf0ee6f None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:26.780612 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c891f5b-9333-454c-8e76-ca9c0cf0ee6f None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:26.780718 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c891f5b-9333-454c-8e76-ca9c0cf0ee6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:26.799616 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-4c891f5b-9333-454c-8e76-ca9c0cf0ee6f None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:04:26.800889 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-4c891f5b-9333-454c-8e76-ca9c0cf0ee6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:26.862920 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1019/2020] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:04:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:04:26.871304 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ac5fe-e602-42b0-9c87-a46c5a33968f None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:26.871633 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ac5fe-e602-42b0-9c87-a46c5a33968f None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:26.871919 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ac5fe-e602-42b0-9c87-a46c5a33968f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:26.890294 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-cd5ac5fe-e602-42b0-9c87-a46c5a33968f None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:04:26.891564 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-cd5ac5fe-e602-42b0-9c87-a46c5a33968f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:26.899288 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1002/2021] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:04:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:04:26.903713 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c891f5b-9333-454c-8e76-ca9c0cf0ee6f None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:26.913229 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c891f5b-9333-454c-8e76-ca9c0cf0ee6f None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:26.918506 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c6b8c-4c91-439c-b5e2-1ec7f932c082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:26.919148 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c6b8c-4c91-439c-b5e2-1ec7f932c082 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:26.919458 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c6b8c-4c91-439c-b5e2-1ec7f932c082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:26.919739 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c6b8c-4c91-439c-b5e2-1ec7f932c082 admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:26.920481 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8c6b8c-4c91-439c-b5e2-1ec7f932c082 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:04:26.920787 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8c6b8c-4c91-439c-b5e2-1ec7f932c082 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:26.922375 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8c6b8c-4c91-439c-b5e2-1ec7f932c082 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:26.934568 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1020/2022] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:04:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:04:26.938701 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ac5fe-e602-42b0-9c87-a46c5a33968f None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:26.948337 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ac5fe-e602-42b0-9c87-a46c5a33968f None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:26.953234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7c8faa-f621-4f26-8eec-d00d901af800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:26.953915 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c8faa-f621-4f26-8eec-d00d901af800 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:26.954194 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c8faa-f621-4f26-8eec-d00d901af800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:26.954469 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c8faa-f621-4f26-8eec-d00d901af800 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:26.955165 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7c8faa-f621-4f26-8eec-d00d901af800 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:26.957072 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7c8faa-f621-4f26-8eec-d00d901af800 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:26.983083 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1003/2023] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:04:26 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:04:26.987238 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c6b8c-4c91-439c-b5e2-1ec7f932c082 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:26.997701 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c6b8c-4c91-439c-b5e2-1ec7f932c082 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:27.023156 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:04:27.055409 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c5319b-98aa-4074-b607-1ab27588be95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:27.056176 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c5319b-98aa-4074-b607-1ab27588be95 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:27.057003 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c5319b-98aa-4074-b607-1ab27588be95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:27.057276 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c5319b-98aa-4074-b607-1ab27588be95 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:27.057967 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c5319b-98aa-4074-b607-1ab27588be95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:27.059925 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c5319b-98aa-4074-b607-1ab27588be95 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:27.078748 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1021/2024] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:04:26 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:04:27.083023 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7c8faa-f621-4f26-8eec-d00d901af800 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:27.093201 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7c8faa-f621-4f26-8eec-d00d901af800 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:27.097800 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcb822a-5dcc-4a63-94d8-65f87bc04bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:27.098456 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb822a-5dcc-4a63-94d8-65f87bc04bec admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:27.098738 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb822a-5dcc-4a63-94d8-65f87bc04bec admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:27.099012 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb822a-5dcc-4a63-94d8-65f87bc04bec admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:27.099762 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcb822a-5dcc-4a63-94d8-65f87bc04bec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:27.101594 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcb822a-5dcc-4a63-94d8-65f87bc04bec admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:27.116671 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1004/2025] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:04:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:04:27.117815 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c5319b-98aa-4074-b607-1ab27588be95 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:27.128471 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c5319b-98aa-4074-b607-1ab27588be95 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:27.133127 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f8ee87-8c1b-42c7-8b47-79a04fa072c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:27.133786 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f8ee87-8c1b-42c7-8b47-79a04fa072c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:27.134075 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f8ee87-8c1b-42c7-8b47-79a04fa072c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:27.134448 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f8ee87-8c1b-42c7-8b47-79a04fa072c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:27.135199 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f8ee87-8c1b-42c7-8b47-79a04fa072c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:27.136823 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f8ee87-8c1b-42c7-8b47-79a04fa072c2 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:27.146565 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1022/2026] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:04:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:04:27.151617 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcb822a-5dcc-4a63-94d8-65f87bc04bec admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:27.161535 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcb822a-5dcc-4a63-94d8-65f87bc04bec admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:27.167186 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8ea269-556a-4df8-b8f0-795985059959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:27.168086 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ea269-556a-4df8-b8f0-795985059959 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:27.168376 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ea269-556a-4df8-b8f0-795985059959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:27.168676 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ea269-556a-4df8-b8f0-795985059959 admin admin] PATH_INFO: `/v3/projects/c261719c878d4dc797bc3c7083ba6618/users/187d8da07ff642238c378cd44672f421/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:27.176711 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:04:27.185165 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8ea269-556a-4df8-b8f0-795985059959 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c261719c878d4dc797bc3c7083ba6618, user_id=187d8da07ff642238c378cd44672f421, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:27.188297 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8ea269-556a-4df8-b8f0-795985059959 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:27.201142 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1005/2027] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:04:27 2022] PUT /identity/v3/projects/c261719c878d4dc797bc3c7083ba6618/users/187d8da07ff642238c378cd44672f421/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:04:27.211207 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a0351b-fd12-44ab-aef5-a9e4461c2aca None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:27.211532 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a0351b-fd12-44ab-aef5-a9e4461c2aca None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:27.211819 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a0351b-fd12-44ab-aef5-a9e4461c2aca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:27.238196 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-56a0351b-fd12-44ab-aef5-a9e4461c2aca None None] MFA Rules not processed for user `187d8da07ff642238c378cd44672f421`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:04:27.239438 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-56a0351b-fd12-44ab-aef5-a9e4461c2aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:27.302280 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1023/2028] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:04:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:04:27.313505 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8ea269-556a-4df8-b8f0-795985059959 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:27.315244 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-cc8ea269-556a-4df8-b8f0-795985059959 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:27.372837 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8ea269-556a-4df8-b8f0-795985059959 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:27.376423 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde415d2-2287-4dc6-b004-1da473bed9ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:27.377073 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde415d2-2287-4dc6-b004-1da473bed9ae service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:27.377361 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde415d2-2287-4dc6-b004-1da473bed9ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:27.377643 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde415d2-2287-4dc6-b004-1da473bed9ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:27.386776 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde415d2-2287-4dc6-b004-1da473bed9ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:27.388232 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde415d2-2287-4dc6-b004-1da473bed9ae service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:27.396915 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1006/2029] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:04:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:04:34.597497 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a0351b-fd12-44ab-aef5-a9e4461c2aca None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:34.659901 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a0351b-fd12-44ab-aef5-a9e4461c2aca None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:34.664314 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07670c3-2da3-48aa-adc6-433c2573e269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:34.665180 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07670c3-2da3-48aa-adc6-433c2573e269 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:34.666013 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07670c3-2da3-48aa-adc6-433c2573e269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:34.666295 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07670c3-2da3-48aa-adc6-433c2573e269 admin admin] PATH_INFO: `/v3/users/187d8da07ff642238c378cd44672f421` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:34.667526 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07670c3-2da3-48aa-adc6-433c2573e269 admin admin] RBAC: Authorizing `identity:delete_user(user_id=187d8da07ff642238c378cd44672f421)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:34.669417 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07670c3-2da3-48aa-adc6-433c2573e269 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:34.709431 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:04:34.716102 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a07670c3-2da3-48aa-adc6-433c2573e269 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '187d8da07ff642238c378cd44672f421'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:34.723927 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a07670c3-2da3-48aa-adc6-433c2573e269 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '187d8da07ff642238c378cd44672f421'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:34.736383 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a07670c3-2da3-48aa-adc6-433c2573e269 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '187d8da07ff642238c378cd44672f421'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:34.737115 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a07670c3-2da3-48aa-adc6-433c2573e269 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '187d8da07ff642238c378cd44672f421'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:34.737723 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a07670c3-2da3-48aa-adc6-433c2573e269 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '187d8da07ff642238c378cd44672f421'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:34.750967 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1024/2030] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:04:34 2022] DELETE /identity/v3/users/187d8da07ff642238c378cd44672f421 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:04:34.755838 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee1f3b6-b724-492e-985f-c1534ceef156 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:34.756622 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee1f3b6-b724-492e-985f-c1534ceef156 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:34.757037 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee1f3b6-b724-492e-985f-c1534ceef156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:34.776534 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-8ee1f3b6-b724-492e-985f-c1534ceef156 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:04:34.777769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-8ee1f3b6-b724-492e-985f-c1534ceef156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:34.838007 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1007/2031] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:04:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:04:34.847420 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07670c3-2da3-48aa-adc6-433c2573e269 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:34.849412 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a07670c3-2da3-48aa-adc6-433c2573e269 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:34.907161 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07670c3-2da3-48aa-adc6-433c2573e269 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:34.910824 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686a262b-f003-4481-94bf-3c33f5a038c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:34.911503 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a262b-f003-4481-94bf-3c33f5a038c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:34.911789 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a262b-f003-4481-94bf-3c33f5a038c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:34.912060 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a262b-f003-4481-94bf-3c33f5a038c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:34.923143 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686a262b-f003-4481-94bf-3c33f5a038c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:34.924626 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686a262b-f003-4481-94bf-3c33f5a038c7 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:34.934903 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1025/2032] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:04:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:04:35.297885 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee1f3b6-b724-492e-985f-c1534ceef156 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:35.299599 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-8ee1f3b6-b724-492e-985f-c1534ceef156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:35.313528 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee1f3b6-b724-492e-985f-c1534ceef156 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:35.318466 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679183e1-e943-49c7-8681-54c6d9994899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:35.319354 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679183e1-e943-49c7-8681-54c6d9994899 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:35.319650 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679183e1-e943-49c7-8681-54c6d9994899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:35.319924 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679183e1-e943-49c7-8681-54c6d9994899 admin admin] PATH_INFO: `/v3/projects/c261719c878d4dc797bc3c7083ba6618` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:35.321123 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679183e1-e943-49c7-8681-54c6d9994899 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c261719c878d4dc797bc3c7083ba6618)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:35.323514 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679183e1-e943-49c7-8681-54c6d9994899 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:35.382407 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-679183e1-e943-49c7-8681-54c6d9994899 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c261719c878d4dc797bc3c7083ba6618', 'request_id': 'req-679183e1-e943-49c7-8681-54c6d9994899'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:35.391915 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-679183e1-e943-49c7-8681-54c6d9994899 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c261719c878d4dc797bc3c7083ba6618', 'request_id': 'req-679183e1-e943-49c7-8681-54c6d9994899'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:35.400260 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-679183e1-e943-49c7-8681-54c6d9994899 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c261719c878d4dc797bc3c7083ba6618', 'request_id': 'req-679183e1-e943-49c7-8681-54c6d9994899'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:35.413103 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-679183e1-e943-49c7-8681-54c6d9994899 admin admin] The token cache is being invalidate because project c261719c878d4dc797bc3c7083ba6618 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:04:35.413555 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-679183e1-e943-49c7-8681-54c6d9994899 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:35.414472 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-679183e1-e943-49c7-8681-54c6d9994899 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:35.417635 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1008/2033] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:04:35 2022] DELETE /identity/v3/projects/c261719c878d4dc797bc3c7083ba6618 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:04:35.424005 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f556bcd-4883-4129-94b6-38ff1ab0d9f7 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:35.424369 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f556bcd-4883-4129-94b6-38ff1ab0d9f7 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:35.425262 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f556bcd-4883-4129-94b6-38ff1ab0d9f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:35.449393 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-2f556bcd-4883-4129-94b6-38ff1ab0d9f7 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:04:35.452356 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-2f556bcd-4883-4129-94b6-38ff1ab0d9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:35.584399 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1026/2034] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:04:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:04:35.593652 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db7a39-a309-4441-b1dd-b602ee282860 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:35.593942 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db7a39-a309-4441-b1dd-b602ee282860 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:35.594221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db7a39-a309-4441-b1dd-b602ee282860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:35.617576 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-15db7a39-a309-4441-b1dd-b602ee282860 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:04:35.619235 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-15db7a39-a309-4441-b1dd-b602ee282860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:35.626656 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1009/2035] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:04:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:04:35.631100 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f556bcd-4883-4129-94b6-38ff1ab0d9f7 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:35.641396 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f556bcd-4883-4129-94b6-38ff1ab0d9f7 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:35.645964 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d51dc6-9d33-492d-8bb6-5381db484ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:35.646620 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d51dc6-9d33-492d-8bb6-5381db484ff2 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:35.646902 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d51dc6-9d33-492d-8bb6-5381db484ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:35.647176 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d51dc6-9d33-492d-8bb6-5381db484ff2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:35.647940 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d51dc6-9d33-492d-8bb6-5381db484ff2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:04:35.648294 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d51dc6-9d33-492d-8bb6-5381db484ff2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:35.650445 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d51dc6-9d33-492d-8bb6-5381db484ff2 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:35.662288 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1027/2036] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:04:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:04:35.667247 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15db7a39-a309-4441-b1dd-b602ee282860 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:35.676800 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15db7a39-a309-4441-b1dd-b602ee282860 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:35.681696 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0583b250-c466-4454-8644-1ffb7ee21879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:35.682346 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583b250-c466-4454-8644-1ffb7ee21879 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:35.682625 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583b250-c466-4454-8644-1ffb7ee21879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:35.682900 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583b250-c466-4454-8644-1ffb7ee21879 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:35.683639 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0583b250-c466-4454-8644-1ffb7ee21879 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:35.685505 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0583b250-c466-4454-8644-1ffb7ee21879 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:35.688740 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:04:35.712880 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1010/2037] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:04:35 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:04:35.717055 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d51dc6-9d33-492d-8bb6-5381db484ff2 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:35.727184 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d51dc6-9d33-492d-8bb6-5381db484ff2 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:35.773078 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:04:35.787608 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc617715-ac1d-4bb3-a1aa-bfd74660851a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:35.788280 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc617715-ac1d-4bb3-a1aa-bfd74660851a admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:35.788564 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc617715-ac1d-4bb3-a1aa-bfd74660851a admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:35.788839 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc617715-ac1d-4bb3-a1aa-bfd74660851a admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:35.789525 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc617715-ac1d-4bb3-a1aa-bfd74660851a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:35.792453 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc617715-ac1d-4bb3-a1aa-bfd74660851a admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:35.814920 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1028/2038] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:04:35 2022] POST /identity/v3/users => generated 284 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:04:35.816148 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0583b250-c466-4454-8644-1ffb7ee21879 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:35.826460 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0583b250-c466-4454-8644-1ffb7ee21879 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:35.831136 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edef771-0f5e-4b57-8103-bc387ff21ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:35.831835 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edef771-0f5e-4b57-8103-bc387ff21ae7 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:35.832119 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edef771-0f5e-4b57-8103-bc387ff21ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:35.832400 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edef771-0f5e-4b57-8103-bc387ff21ae7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:35.833147 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edef771-0f5e-4b57-8103-bc387ff21ae7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:35.834714 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edef771-0f5e-4b57-8103-bc387ff21ae7 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:35.844845 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1011/2039] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:04:35.848717 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc617715-ac1d-4bb3-a1aa-bfd74660851a admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:35.859261 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc617715-ac1d-4bb3-a1aa-bfd74660851a admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:35.863837 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44416c7d-48ff-45a7-b160-0e6a6e3d813b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:35.864478 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44416c7d-48ff-45a7-b160-0e6a6e3d813b admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:35.864758 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44416c7d-48ff-45a7-b160-0e6a6e3d813b admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:35.865027 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44416c7d-48ff-45a7-b160-0e6a6e3d813b admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:35.865741 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44416c7d-48ff-45a7-b160-0e6a6e3d813b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:35.867303 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44416c7d-48ff-45a7-b160-0e6a6e3d813b admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:35.881695 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1029/2040] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:04:35.883355 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edef771-0f5e-4b57-8103-bc387ff21ae7 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:35.893781 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edef771-0f5e-4b57-8103-bc387ff21ae7 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:35.898388 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b872c6d7-9fa8-486a-bddb-6035facca642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:35.899273 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b872c6d7-9fa8-486a-bddb-6035facca642 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:35.899623 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b872c6d7-9fa8-486a-bddb-6035facca642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:35.899962 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b872c6d7-9fa8-486a-bddb-6035facca642 admin admin] PATH_INFO: `/v3/projects/8b3b4052916c453788c44757afb0f156/users/c413d005197847daa8c7f789cdfeadaa/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:35.914444 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b872c6d7-9fa8-486a-bddb-6035facca642 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b3b4052916c453788c44757afb0f156, user_id=c413d005197847daa8c7f789cdfeadaa, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:35.917611 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b872c6d7-9fa8-486a-bddb-6035facca642 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:35.933355 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1012/2041] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:04:35 2022] PUT /identity/v3/projects/8b3b4052916c453788c44757afb0f156/users/c413d005197847daa8c7f789cdfeadaa/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:04:35.946127 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac147f2-9683-4592-82bb-b066d292676d None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:35.946418 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac147f2-9683-4592-82bb-b066d292676d None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:35.946744 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac147f2-9683-4592-82bb-b066d292676d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:35.975468 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-bac147f2-9683-4592-82bb-b066d292676d None None] MFA Rules not processed for user `c413d005197847daa8c7f789cdfeadaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:04:35.976978 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-bac147f2-9683-4592-82bb-b066d292676d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:36.045919 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1030/2042] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:04:35 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:04:36.057654 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b872c6d7-9fa8-486a-bddb-6035facca642 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:36.059480 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-b872c6d7-9fa8-486a-bddb-6035facca642 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:36.118020 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b872c6d7-9fa8-486a-bddb-6035facca642 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:36.122575 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa156e2f-5ea0-45f6-b9c3-0af5f8337c20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:36.123244 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa156e2f-5ea0-45f6-b9c3-0af5f8337c20 service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:36.123558 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa156e2f-5ea0-45f6-b9c3-0af5f8337c20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:36.123844 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa156e2f-5ea0-45f6-b9c3-0af5f8337c20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:36.133296 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa156e2f-5ea0-45f6-b9c3-0af5f8337c20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:36.134879 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa156e2f-5ea0-45f6-b9c3-0af5f8337c20 service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:36.145496 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1013/2043] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:04:36 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:04:36.173817 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac147f2-9683-4592-82bb-b066d292676d None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:36.281146 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac147f2-9683-4592-82bb-b066d292676d None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:36.287095 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6950224e-d0b1-49c5-a240-39442d2e566e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:36.287725 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6950224e-d0b1-49c5-a240-39442d2e566e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:36.288014 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6950224e-d0b1-49c5-a240-39442d2e566e admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:36.288355 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6950224e-d0b1-49c5-a240-39442d2e566e admin admin] PATH_INFO: `/v3/users/c413d005197847daa8c7f789cdfeadaa` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:36.289836 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6950224e-d0b1-49c5-a240-39442d2e566e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c413d005197847daa8c7f789cdfeadaa)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:36.293141 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6950224e-d0b1-49c5-a240-39442d2e566e admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:36.380792 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-6950224e-d0b1-49c5-a240-39442d2e566e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c413d005197847daa8c7f789cdfeadaa'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:36.391833 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-6950224e-d0b1-49c5-a240-39442d2e566e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c413d005197847daa8c7f789cdfeadaa'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:36.393138 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:04:36.407446 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-6950224e-d0b1-49c5-a240-39442d2e566e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c413d005197847daa8c7f789cdfeadaa'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:36.408205 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-6950224e-d0b1-49c5-a240-39442d2e566e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c413d005197847daa8c7f789cdfeadaa'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:36.408905 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-6950224e-d0b1-49c5-a240-39442d2e566e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c413d005197847daa8c7f789cdfeadaa'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:36.425651 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1031/2044] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:04:36 2022] DELETE /identity/v3/users/c413d005197847daa8c7f789cdfeadaa => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:04:36.430399 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553c60e-d2dd-4802-aa79-99c608bb98ed None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:36.430541 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553c60e-d2dd-4802-aa79-99c608bb98ed None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:36.431367 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553c60e-d2dd-4802-aa79-99c608bb98ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:36.452312 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-0553c60e-d2dd-4802-aa79-99c608bb98ed None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:04:36.454368 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-0553c60e-d2dd-4802-aa79-99c608bb98ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:36.542229 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1014/2045] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:04:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:04:36.556802 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6950224e-d0b1-49c5-a240-39442d2e566e admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:36.558656 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-6950224e-d0b1-49c5-a240-39442d2e566e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:36.646717 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6950224e-d0b1-49c5-a240-39442d2e566e admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:36.653359 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2d0fd6-6f9d-4fe0-90a4-85a3d0b963ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:36.654014 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d0fd6-6f9d-4fe0-90a4-85a3d0b963ac service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:36.654291 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d0fd6-6f9d-4fe0-90a4-85a3d0b963ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:36.655009 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d0fd6-6f9d-4fe0-90a4-85a3d0b963ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:36.664998 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2d0fd6-6f9d-4fe0-90a4-85a3d0b963ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:36.666338 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2d0fd6-6f9d-4fe0-90a4-85a3d0b963ac service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:36.676045 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1032/2046] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:04:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:04:36.964651 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0553c60e-d2dd-4802-aa79-99c608bb98ed None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:36.966157 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-0553c60e-d2dd-4802-aa79-99c608bb98ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:36.978559 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0553c60e-d2dd-4802-aa79-99c608bb98ed None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:36.994276 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf9c4f6-5507-4eea-a96e-5fe359669699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:36.995185 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf9c4f6-5507-4eea-a96e-5fe359669699 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:36.995502 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf9c4f6-5507-4eea-a96e-5fe359669699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:36.995780 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf9c4f6-5507-4eea-a96e-5fe359669699 admin admin] PATH_INFO: `/v3/projects/8b3b4052916c453788c44757afb0f156` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:36.997362 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf9c4f6-5507-4eea-a96e-5fe359669699 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b3b4052916c453788c44757afb0f156)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:36.999263 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf9c4f6-5507-4eea-a96e-5fe359669699 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:37.051252 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:04:37.052660 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-ecf9c4f6-5507-4eea-a96e-5fe359669699 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b3b4052916c453788c44757afb0f156', 'request_id': 'req-ecf9c4f6-5507-4eea-a96e-5fe359669699'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:37.062435 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-ecf9c4f6-5507-4eea-a96e-5fe359669699 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b3b4052916c453788c44757afb0f156', 'request_id': 'req-ecf9c4f6-5507-4eea-a96e-5fe359669699'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:37.069777 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-ecf9c4f6-5507-4eea-a96e-5fe359669699 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b3b4052916c453788c44757afb0f156', 'request_id': 'req-ecf9c4f6-5507-4eea-a96e-5fe359669699'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:37.081231 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-ecf9c4f6-5507-4eea-a96e-5fe359669699 admin admin] The token cache is being invalidate because project 8b3b4052916c453788c44757afb0f156 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:04:37.081520 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-ecf9c4f6-5507-4eea-a96e-5fe359669699 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:37.082169 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-ecf9c4f6-5507-4eea-a96e-5fe359669699 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:37.083999 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1015/2047] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:04:36 2022] DELETE /identity/v3/projects/8b3b4052916c453788c44757afb0f156 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:04:37.089063 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0fdfd4-775d-4b24-8d57-f5d85014904a None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:37.089427 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0fdfd4-775d-4b24-8d57-f5d85014904a None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:37.089715 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0fdfd4-775d-4b24-8d57-f5d85014904a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:37.108576 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-ef0fdfd4-775d-4b24-8d57-f5d85014904a None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:04:37.109793 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-ef0fdfd4-775d-4b24-8d57-f5d85014904a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:37.171128 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1033/2048] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:04:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:04:37.179616 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e350378-2df3-4c61-a912-c4fb3d063a04 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:37.180099 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e350378-2df3-4c61-a912-c4fb3d063a04 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:37.180381 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e350378-2df3-4c61-a912-c4fb3d063a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:37.200074 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-2e350378-2df3-4c61-a912-c4fb3d063a04 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:04:37.201339 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-2e350378-2df3-4c61-a912-c4fb3d063a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:37.209435 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1016/2049] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:04:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:04:37.213724 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0fdfd4-775d-4b24-8d57-f5d85014904a None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:37.223380 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0fdfd4-775d-4b24-8d57-f5d85014904a None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:37.228636 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0e400c-b729-4813-b371-4a79c0a616b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:37.229347 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e400c-b729-4813-b371-4a79c0a616b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:37.229656 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e400c-b729-4813-b371-4a79c0a616b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:37.229977 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e400c-b729-4813-b371-4a79c0a616b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:37.230782 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0e400c-b729-4813-b371-4a79c0a616b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:04:37.231097 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0e400c-b729-4813-b371-4a79c0a616b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:37.233015 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0e400c-b729-4813-b371-4a79c0a616b7 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:37.252678 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1034/2050] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:04:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:04:37.253963 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e350378-2df3-4c61-a912-c4fb3d063a04 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:37.263982 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e350378-2df3-4c61-a912-c4fb3d063a04 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:37.268611 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d729f9b3-ee79-4522-a12a-bb922cd9bb21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:37.269272 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729f9b3-ee79-4522-a12a-bb922cd9bb21 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:37.269556 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729f9b3-ee79-4522-a12a-bb922cd9bb21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:37.269834 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729f9b3-ee79-4522-a12a-bb922cd9bb21 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:37.270523 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d729f9b3-ee79-4522-a12a-bb922cd9bb21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:37.272490 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d729f9b3-ee79-4522-a12a-bb922cd9bb21 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:37.301590 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1017/2051] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:04:37 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:04:37.304498 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0e400c-b729-4813-b371-4a79c0a616b7 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:37.315194 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0e400c-b729-4813-b371-4a79c0a616b7 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:37.382322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b6788-b012-41c8-8dcd-f068e8fd66f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:37.383062 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b6788-b012-41c8-8dcd-f068e8fd66f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:37.383462 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b6788-b012-41c8-8dcd-f068e8fd66f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:37.383744 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b6788-b012-41c8-8dcd-f068e8fd66f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:37.385017 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539b6788-b012-41c8-8dcd-f068e8fd66f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:37.386980 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539b6788-b012-41c8-8dcd-f068e8fd66f1 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:37.396512 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:04:37.412051 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1035/2052] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:04:37 2022] POST /identity/v3/users => generated 287 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:04:37.417403 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d729f9b3-ee79-4522-a12a-bb922cd9bb21 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:37.429162 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d729f9b3-ee79-4522-a12a-bb922cd9bb21 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:37.434908 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f96aa5-e8db-48b6-b13e-5f0395330bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:37.435751 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f96aa5-e8db-48b6-b13e-5f0395330bb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:37.436055 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f96aa5-e8db-48b6-b13e-5f0395330bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:37.436326 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f96aa5-e8db-48b6-b13e-5f0395330bb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:37.437069 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f96aa5-e8db-48b6-b13e-5f0395330bb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:37.438684 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f96aa5-e8db-48b6-b13e-5f0395330bb2 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:37.453542 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1018/2053] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:04:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:04:37.457559 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b6788-b012-41c8-8dcd-f068e8fd66f1 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:37.482767 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b6788-b012-41c8-8dcd-f068e8fd66f1 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:37.496144 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b8d11-07b9-4c3c-b271-b591a0750463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:37.497027 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b8d11-07b9-4c3c-b271-b591a0750463 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:37.497445 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b8d11-07b9-4c3c-b271-b591a0750463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:37.497840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b8d11-07b9-4c3c-b271-b591a0750463 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:37.498776 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323b8d11-07b9-4c3c-b271-b591a0750463 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:37.500705 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323b8d11-07b9-4c3c-b271-b591a0750463 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:37.514188 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1036/2054] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:04:37 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:04:37.518783 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f96aa5-e8db-48b6-b13e-5f0395330bb2 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:37.534110 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f96aa5-e8db-48b6-b13e-5f0395330bb2 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:37.547920 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934a3cc-034e-4375-8eba-2d8d71a46f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:37.547920 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934a3cc-034e-4375-8eba-2d8d71a46f3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:37.548270 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934a3cc-034e-4375-8eba-2d8d71a46f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:37.548447 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934a3cc-034e-4375-8eba-2d8d71a46f3c admin admin] PATH_INFO: `/v3/projects/f240f9b198124b58826697af9f8ab205/users/d03864707f284a14ad9726c4023f32b8/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:37.593321 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3934a3cc-034e-4375-8eba-2d8d71a46f3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f240f9b198124b58826697af9f8ab205, user_id=d03864707f284a14ad9726c4023f32b8, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:37.596221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3934a3cc-034e-4375-8eba-2d8d71a46f3c admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:37.612812 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1019/2055] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:04:37 2022] PUT /identity/v3/projects/f240f9b198124b58826697af9f8ab205/users/d03864707f284a14ad9726c4023f32b8/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:04:37.624246 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2f5aa-d47f-4e70-885e-8bda9ec95adf None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:37.624534 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2f5aa-d47f-4e70-885e-8bda9ec95adf None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:37.624811 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2f5aa-d47f-4e70-885e-8bda9ec95adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:37.662561 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-bfd2f5aa-d47f-4e70-885e-8bda9ec95adf None None] MFA Rules not processed for user `d03864707f284a14ad9726c4023f32b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:04:37.663394 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-bfd2f5aa-d47f-4e70-885e-8bda9ec95adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:37.761737 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1037/2056] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:04:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:04:37.770046 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934a3cc-034e-4375-8eba-2d8d71a46f3c admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:37.772010 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-3934a3cc-034e-4375-8eba-2d8d71a46f3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:37.856272 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934a3cc-034e-4375-8eba-2d8d71a46f3c admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:37.860370 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744e3a56-fcc3-4547-9509-b30f798ff6ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:37.861307 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e3a56-fcc3-4547-9509-b30f798ff6ec service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:37.861540 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e3a56-fcc3-4547-9509-b30f798ff6ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:37.861921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e3a56-fcc3-4547-9509-b30f798ff6ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:37.880081 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744e3a56-fcc3-4547-9509-b30f798ff6ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:37.882126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744e3a56-fcc3-4547-9509-b30f798ff6ec service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:37.896681 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1020/2057] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:04:37 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:04:43.566081 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8bbec-1702-487d-95b1-f550e2066ff2 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:43.566539 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8bbec-1702-487d-95b1-f550e2066ff2 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:43.566632 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8bbec-1702-487d-95b1-f550e2066ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:43.585258 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-54b8bbec-1702-487d-95b1-f550e2066ff2 None None] MFA Rules not processed for user `d03864707f284a14ad9726c4023f32b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:04:43.586471 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-54b8bbec-1702-487d-95b1-f550e2066ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:43.594419 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1038/2058] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:04:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:04:46.998286 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744e3a56-fcc3-4547-9509-b30f798ff6ec service cinder] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:46.999982 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-744e3a56-fcc3-4547-9509-b30f798ff6ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:47.067290 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744e3a56-fcc3-4547-9509-b30f798ff6ec service cinder] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:47.072447 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9659ff05-7df7-4a2c-92cd-5ffa334226d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:47.073308 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9659ff05-7df7-4a2c-92cd-5ffa334226d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:47.073589 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9659ff05-7df7-4a2c-92cd-5ffa334226d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:47.073860 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9659ff05-7df7-4a2c-92cd-5ffa334226d3 admin admin] PATH_INFO: `/v3/users/661659f9e2b548ac9fd4fd2a3ca58ad5` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:47.075045 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9659ff05-7df7-4a2c-92cd-5ffa334226d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=661659f9e2b548ac9fd4fd2a3ca58ad5)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:47.076984 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9659ff05-7df7-4a2c-92cd-5ffa334226d3 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:47.093144 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:04:47.128270 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-9659ff05-7df7-4a2c-92cd-5ffa334226d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '661659f9e2b548ac9fd4fd2a3ca58ad5'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:47.128949 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-9659ff05-7df7-4a2c-92cd-5ffa334226d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '661659f9e2b548ac9fd4fd2a3ca58ad5'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:47.137106 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-9659ff05-7df7-4a2c-92cd-5ffa334226d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '661659f9e2b548ac9fd4fd2a3ca58ad5'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:47.154348 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-9659ff05-7df7-4a2c-92cd-5ffa334226d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '661659f9e2b548ac9fd4fd2a3ca58ad5'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:47.161289 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-9659ff05-7df7-4a2c-92cd-5ffa334226d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '661659f9e2b548ac9fd4fd2a3ca58ad5'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:47.167475 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1021/2059] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:04:46 2022] DELETE /identity/v3/users/661659f9e2b548ac9fd4fd2a3ca58ad5 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:04:47.171974 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f408eb-de5a-4a44-81ed-fb1bc2abe767 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:47.172456 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f408eb-de5a-4a44-81ed-fb1bc2abe767 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:47.172555 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f408eb-de5a-4a44-81ed-fb1bc2abe767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:47.191094 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-18f408eb-de5a-4a44-81ed-fb1bc2abe767 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:04:47.192324 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-18f408eb-de5a-4a44-81ed-fb1bc2abe767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:47.261502 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1039/2060] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:04:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:04:47.292747 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9659ff05-7df7-4a2c-92cd-5ffa334226d3 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:47.294298 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-9659ff05-7df7-4a2c-92cd-5ffa334226d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:47.357357 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9659ff05-7df7-4a2c-92cd-5ffa334226d3 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:47.361023 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa9bd86-3a02-49ce-a911-db7a9bf9cab4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:47.361675 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa9bd86-3a02-49ce-a911-db7a9bf9cab4 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:47.361962 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa9bd86-3a02-49ce-a911-db7a9bf9cab4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:47.362243 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa9bd86-3a02-49ce-a911-db7a9bf9cab4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:47.371807 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa9bd86-3a02-49ce-a911-db7a9bf9cab4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:47.373228 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa9bd86-3a02-49ce-a911-db7a9bf9cab4 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:47.382524 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1022/2061] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:04:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:04:47.632384 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f408eb-de5a-4a44-81ed-fb1bc2abe767 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:47.634356 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-18f408eb-de5a-4a44-81ed-fb1bc2abe767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:47.646051 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f408eb-de5a-4a44-81ed-fb1bc2abe767 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:47.651531 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8d16ed-908f-4c14-a4fc-9d924838561d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:47.652494 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d16ed-908f-4c14-a4fc-9d924838561d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:47.652682 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d16ed-908f-4c14-a4fc-9d924838561d admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:47.652950 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d16ed-908f-4c14-a4fc-9d924838561d admin admin] PATH_INFO: `/v3/users/42b3410fb02f44748ce4afbf3e911a45` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:47.654140 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8d16ed-908f-4c14-a4fc-9d924838561d admin admin] RBAC: Authorizing `identity:delete_user(user_id=42b3410fb02f44748ce4afbf3e911a45)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:47.656110 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8d16ed-908f-4c14-a4fc-9d924838561d admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:47.739787 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-9a8d16ed-908f-4c14-a4fc-9d924838561d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42b3410fb02f44748ce4afbf3e911a45'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:47.751457 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-9a8d16ed-908f-4c14-a4fc-9d924838561d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42b3410fb02f44748ce4afbf3e911a45'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:47.769312 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:INSERT=1,keystone:SELECT=92,keystone:DELETE=7 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:04:47.782740 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-9a8d16ed-908f-4c14-a4fc-9d924838561d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42b3410fb02f44748ce4afbf3e911a45'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:47.784334 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-9a8d16ed-908f-4c14-a4fc-9d924838561d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42b3410fb02f44748ce4afbf3e911a45'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:47.785163 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-9a8d16ed-908f-4c14-a4fc-9d924838561d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42b3410fb02f44748ce4afbf3e911a45'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:47.808082 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1040/2062] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:04:47 2022] DELETE /identity/v3/users/42b3410fb02f44748ce4afbf3e911a45 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:04:47.813938 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c67fee3-3b49-41d5-af3a-d36f8a1fb47e None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:47.814668 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c67fee3-3b49-41d5-af3a-d36f8a1fb47e None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:47.815849 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c67fee3-3b49-41d5-af3a-d36f8a1fb47e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:47.842239 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-1c67fee3-3b49-41d5-af3a-d36f8a1fb47e None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:04:47.843797 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-1c67fee3-3b49-41d5-af3a-d36f8a1fb47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:47.926071 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1023/2063] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:04:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:04:47.950014 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8d16ed-908f-4c14-a4fc-9d924838561d admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:47.952143 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-9a8d16ed-908f-4c14-a4fc-9d924838561d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:48.030440 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c67fee3-3b49-41d5-af3a-d36f8a1fb47e None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:48.032085 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-1c67fee3-3b49-41d5-af3a-d36f8a1fb47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:48.053508 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c67fee3-3b49-41d5-af3a-d36f8a1fb47e None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:48.066809 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab3268-2b09-4c84-96ba-7df3f9b0fdb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:48.067863 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab3268-2b09-4c84-96ba-7df3f9b0fdb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:48.068627 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab3268-2b09-4c84-96ba-7df3f9b0fdb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:48.068988 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab3268-2b09-4c84-96ba-7df3f9b0fdb8 admin admin] PATH_INFO: `/v3/projects/bddab25e796243688fff05c9b2d4af31` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:48.070339 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab3268-2b09-4c84-96ba-7df3f9b0fdb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bddab25e796243688fff05c9b2d4af31)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:48.071504 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8d16ed-908f-4c14-a4fc-9d924838561d admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:48.072485 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab3268-2b09-4c84-96ba-7df3f9b0fdb8 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:48.075278 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1a9df9-f649-4968-918c-dc93b68b105f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:48.076168 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a9df9-f649-4968-918c-dc93b68b105f service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:48.076472 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a9df9-f649-4968-918c-dc93b68b105f service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:48.076712 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a9df9-f649-4968-918c-dc93b68b105f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:48.088155 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1a9df9-f649-4968-918c-dc93b68b105f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:48.089627 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1a9df9-f649-4968-918c-dc93b68b105f service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:48.103610 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1041/2064] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:04:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 22:04:48.116023 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:04:48.135783 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d1ab3268-2b09-4c84-96ba-7df3f9b0fdb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bddab25e796243688fff05c9b2d4af31', 'request_id': 'req-d1ab3268-2b09-4c84-96ba-7df3f9b0fdb8'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:48.146358 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d1ab3268-2b09-4c84-96ba-7df3f9b0fdb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bddab25e796243688fff05c9b2d4af31', 'request_id': 'req-d1ab3268-2b09-4c84-96ba-7df3f9b0fdb8'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:48.157958 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d1ab3268-2b09-4c84-96ba-7df3f9b0fdb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bddab25e796243688fff05c9b2d4af31', 'request_id': 'req-d1ab3268-2b09-4c84-96ba-7df3f9b0fdb8'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:48.175600 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d1ab3268-2b09-4c84-96ba-7df3f9b0fdb8 admin admin] The token cache is being invalidate because project bddab25e796243688fff05c9b2d4af31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:04:48.176006 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d1ab3268-2b09-4c84-96ba-7df3f9b0fdb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:48.176783 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d1ab3268-2b09-4c84-96ba-7df3f9b0fdb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:48.179478 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1024/2065] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:04:48 2022] DELETE /identity/v3/projects/bddab25e796243688fff05c9b2d4af31 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:04:48.184752 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879cc73b-29ff-4eb0-b74f-8abbffb74f16 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:48.185233 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879cc73b-29ff-4eb0-b74f-8abbffb74f16 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:48.185363 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879cc73b-29ff-4eb0-b74f-8abbffb74f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:48.205565 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-879cc73b-29ff-4eb0-b74f-8abbffb74f16 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:04:48.206881 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-879cc73b-29ff-4eb0-b74f-8abbffb74f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:48.318316 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1042/2066] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:04:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:04:48.326303 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e73231-7542-4157-bf4c-d7af4ed106e7 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:48.326665 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e73231-7542-4157-bf4c-d7af4ed106e7 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:48.327007 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e73231-7542-4157-bf4c-d7af4ed106e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:48.348535 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-52e73231-7542-4157-bf4c-d7af4ed106e7 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:04:48.349763 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-52e73231-7542-4157-bf4c-d7af4ed106e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:48.389167 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1025/2067] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:04:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:04:48.394047 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879cc73b-29ff-4eb0-b74f-8abbffb74f16 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:48.407062 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879cc73b-29ff-4eb0-b74f-8abbffb74f16 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:48.412019 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9062dc1b-cd12-425e-afb2-a874224d5752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:48.412688 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062dc1b-cd12-425e-afb2-a874224d5752 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:48.412967 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062dc1b-cd12-425e-afb2-a874224d5752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:48.413244 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062dc1b-cd12-425e-afb2-a874224d5752 admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:48.414185 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9062dc1b-cd12-425e-afb2-a874224d5752 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:04:48.414908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9062dc1b-cd12-425e-afb2-a874224d5752 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:48.416859 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9062dc1b-cd12-425e-afb2-a874224d5752 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:48.431268 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1043/2068] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:04:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:04:48.436176 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e73231-7542-4157-bf4c-d7af4ed106e7 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:48.449072 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e73231-7542-4157-bf4c-d7af4ed106e7 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:48.456504 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9062dc1b-cd12-425e-afb2-a874224d5752 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:48.458504 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-9062dc1b-cd12-425e-afb2-a874224d5752 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:48.470129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0636ad46-b1d0-427c-8a8e-1d6ad7bed5d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:48.470862 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636ad46-b1d0-427c-8a8e-1d6ad7bed5d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:48.472915 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636ad46-b1d0-427c-8a8e-1d6ad7bed5d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:48.472915 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636ad46-b1d0-427c-8a8e-1d6ad7bed5d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:48.473589 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0636ad46-b1d0-427c-8a8e-1d6ad7bed5d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:48.477088 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0636ad46-b1d0-427c-8a8e-1d6ad7bed5d3 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:48.481665 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9062dc1b-cd12-425e-afb2-a874224d5752 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:48.488037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d77c08-9900-4c34-89c5-1ceeea63ee92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:48.489006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d77c08-9900-4c34-89c5-1ceeea63ee92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:48.489407 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d77c08-9900-4c34-89c5-1ceeea63ee92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:48.489788 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d77c08-9900-4c34-89c5-1ceeea63ee92 admin admin] PATH_INFO: `/v3/projects/02bb43fcee824adf8e60dfffcc5258c0` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:48.491267 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d77c08-9900-4c34-89c5-1ceeea63ee92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02bb43fcee824adf8e60dfffcc5258c0)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:48.495027 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d77c08-9900-4c34-89c5-1ceeea63ee92 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:48.521458 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1026/2069] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:04:48 2022] POST /identity/v3/projects => generated 369 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:04:48.526151 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0636ad46-b1d0-427c-8a8e-1d6ad7bed5d3 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:48.556040 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c6d77c08-9900-4c34-89c5-1ceeea63ee92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02bb43fcee824adf8e60dfffcc5258c0', 'request_id': 'req-c6d77c08-9900-4c34-89c5-1ceeea63ee92'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:48.565372 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c6d77c08-9900-4c34-89c5-1ceeea63ee92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02bb43fcee824adf8e60dfffcc5258c0', 'request_id': 'req-c6d77c08-9900-4c34-89c5-1ceeea63ee92'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:48.574839 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:04:48.582991 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c6d77c08-9900-4c34-89c5-1ceeea63ee92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02bb43fcee824adf8e60dfffcc5258c0', 'request_id': 'req-c6d77c08-9900-4c34-89c5-1ceeea63ee92'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:48.602807 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c6d77c08-9900-4c34-89c5-1ceeea63ee92 admin admin] The token cache is being invalidate because project 02bb43fcee824adf8e60dfffcc5258c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:04:48.603702 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c6d77c08-9900-4c34-89c5-1ceeea63ee92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:48.604392 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c6d77c08-9900-4c34-89c5-1ceeea63ee92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:48.605862 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0636ad46-b1d0-427c-8a8e-1d6ad7bed5d3 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:48.606434 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1044/2070] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:04:48 2022] DELETE /identity/v3/projects/02bb43fcee824adf8e60dfffcc5258c0 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:04:48.611302 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa6f01-7ac9-44be-ba04-e09f7ea9454c None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:48.611623 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa6f01-7ac9-44be-ba04-e09f7ea9454c None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:48.612154 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa6f01-7ac9-44be-ba04-e09f7ea9454c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:48.632495 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-97aa6f01-7ac9-44be-ba04-e09f7ea9454c None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:04:48.634239 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-97aa6f01-7ac9-44be-ba04-e09f7ea9454c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:48.680249 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b781f2a-df40-4e16-90d7-981adf754f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:48.680962 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b781f2a-df40-4e16-90d7-981adf754f00 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:48.681249 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b781f2a-df40-4e16-90d7-981adf754f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:48.681524 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b781f2a-df40-4e16-90d7-981adf754f00 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:48.682217 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b781f2a-df40-4e16-90d7-981adf754f00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:48.684171 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b781f2a-df40-4e16-90d7-981adf754f00 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:48.704504 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1027/2071] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:04:48 2022] POST /identity/v3/users => generated 285 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:04:48.705264 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1045/2072] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:04:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:04:48.713193 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b781f2a-df40-4e16-90d7-981adf754f00 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:48.713193 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-9b781f2a-df40-4e16-90d7-981adf754f00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:48.714719 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa1940c-b874-4f3b-b8ed-af84d1198c99 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:48.715013 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa1940c-b874-4f3b-b8ed-af84d1198c99 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:48.715296 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa1940c-b874-4f3b-b8ed-af84d1198c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:48.722048 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b781f2a-df40-4e16-90d7-981adf754f00 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:48.727179 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb7d131-4cf9-4ca6-8bcd-8896cd5dc6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:48.727910 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb7d131-4cf9-4ca6-8bcd-8896cd5dc6ce admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:48.728190 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb7d131-4cf9-4ca6-8bcd-8896cd5dc6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:48.728465 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb7d131-4cf9-4ca6-8bcd-8896cd5dc6ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:48.729192 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb7d131-4cf9-4ca6-8bcd-8896cd5dc6ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:48.730771 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb7d131-4cf9-4ca6-8bcd-8896cd5dc6ce admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:48.734431 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-baa1940c-b874-4f3b-b8ed-af84d1198c99 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:04:48.735667 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-baa1940c-b874-4f3b-b8ed-af84d1198c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:48.740786 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1028/2073] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 22:04:48.744901 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb7d131-4cf9-4ca6-8bcd-8896cd5dc6ce admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:48.746685 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1046/2074] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:04:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:04:48.750615 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa1940c-b874-4f3b-b8ed-af84d1198c99 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:48.754769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb7d131-4cf9-4ca6-8bcd-8896cd5dc6ce admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:48.760419 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d76aa0b-1474-46d4-a479-f03b95cabf65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:48.761085 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d76aa0b-1474-46d4-a479-f03b95cabf65 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:48.761374 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d76aa0b-1474-46d4-a479-f03b95cabf65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:48.761659 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d76aa0b-1474-46d4-a479-f03b95cabf65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:48.761841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa1940c-b874-4f3b-b8ed-af84d1198c99 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:48.762537 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d76aa0b-1474-46d4-a479-f03b95cabf65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:48.768649 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d76aa0b-1474-46d4-a479-f03b95cabf65 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:48.768805 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee33305-d0ac-49f8-b88f-5add51142ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:48.768805 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee33305-d0ac-49f8-b88f-5add51142ff9 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:48.768805 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee33305-d0ac-49f8-b88f-5add51142ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:48.768805 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee33305-d0ac-49f8-b88f-5add51142ff9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:48.769159 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee33305-d0ac-49f8-b88f-5add51142ff9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:04:48.769159 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee33305-d0ac-49f8-b88f-5add51142ff9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:48.770764 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee33305-d0ac-49f8-b88f-5add51142ff9 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:48.776957 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1029/2075] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 22:04:48.782260 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d76aa0b-1474-46d4-a479-f03b95cabf65 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:48.787803 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1047/2076] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:04:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 22:04:48.793631 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee33305-d0ac-49f8-b88f-5add51142ff9 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:48.802591 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d76aa0b-1474-46d4-a479-f03b95cabf65 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:48.803453 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee33305-d0ac-49f8-b88f-5add51142ff9 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:48.809212 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a48d2c6-639b-4acd-aace-544e264aaef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:48.809938 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48d2c6-639b-4acd-aace-544e264aaef0 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:48.810256 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ee544f-cfc1-4c80-8624-4a85cf05dec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:48.810395 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48d2c6-639b-4acd-aace-544e264aaef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:48.811092 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48d2c6-639b-4acd-aace-544e264aaef0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:48.811490 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a48d2c6-639b-4acd-aace-544e264aaef0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:48.812432 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee544f-cfc1-4c80-8624-4a85cf05dec4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:48.812946 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee544f-cfc1-4c80-8624-4a85cf05dec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:48.813384 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a48d2c6-639b-4acd-aace-544e264aaef0 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:48.813506 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee544f-cfc1-4c80-8624-4a85cf05dec4 admin admin] PATH_INFO: `/v3/projects/5d15f11ecfdf49c789f948394783b61e/users/e9955cc2c9fe47a3b0333133706eec7d/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:48.816815 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:04:48.840798 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ee544f-cfc1-4c80-8624-4a85cf05dec4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d15f11ecfdf49c789f948394783b61e, user_id=e9955cc2c9fe47a3b0333133706eec7d, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:48.846081 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1048/2077] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:04:48 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:04:48.846269 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ee544f-cfc1-4c80-8624-4a85cf05dec4 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:48.847352 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a48d2c6-639b-4acd-aace-544e264aaef0 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:48.870616 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1030/2078] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:04:48 2022] PUT /identity/v3/projects/5d15f11ecfdf49c789f948394783b61e/users/e9955cc2c9fe47a3b0333133706eec7d/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:04:48.873448 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2093a123-b5c0-4fa0-92ab-1c060f19bfa9 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:48.873835 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2093a123-b5c0-4fa0-92ab-1c060f19bfa9 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:48.874181 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2093a123-b5c0-4fa0-92ab-1c060f19bfa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:48.904565 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-2093a123-b5c0-4fa0-92ab-1c060f19bfa9 None None] MFA Rules not processed for user `e9955cc2c9fe47a3b0333133706eec7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:04:48.905898 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-2093a123-b5c0-4fa0-92ab-1c060f19bfa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:48.921178 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a48d2c6-639b-4acd-aace-544e264aaef0 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:48.979612 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1031/2079] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:04:48 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:04:48.982430 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:04:48.991722 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2093a123-b5c0-4fa0-92ab-1c060f19bfa9 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:48.991977 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624793da-2891-4b85-b28a-305e91105dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:48.992663 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624793da-2891-4b85-b28a-305e91105dd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:48.993001 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624793da-2891-4b85-b28a-305e91105dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:48.993183 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-2093a123-b5c0-4fa0-92ab-1c060f19bfa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:48.993434 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624793da-2891-4b85-b28a-305e91105dd9 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:48.994255 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624793da-2891-4b85-b28a-305e91105dd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:48.996213 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624793da-2891-4b85-b28a-305e91105dd9 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:49.019731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1049/2080] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:04:48 2022] POST /identity/v3/users => generated 289 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:04:49.020868 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624793da-2891-4b85-b28a-305e91105dd9 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:49.031669 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624793da-2891-4b85-b28a-305e91105dd9 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:49.036707 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e94909-0b7f-43d6-9177-e4e824b47cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:49.037370 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e94909-0b7f-43d6-9177-e4e824b47cf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:49.037775 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e94909-0b7f-43d6-9177-e4e824b47cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:49.038188 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e94909-0b7f-43d6-9177-e4e824b47cf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:49.038928 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e94909-0b7f-43d6-9177-e4e824b47cf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:49.041801 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e94909-0b7f-43d6-9177-e4e824b47cf1 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:49.052974 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1050/2081] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 22:04:49.054891 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2093a123-b5c0-4fa0-92ab-1c060f19bfa9 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:49.057352 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e94909-0b7f-43d6-9177-e4e824b47cf1 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:49.058805 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd3b688-4b07-434b-934e-ee6777f3923d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96987b04471944bc86ab44ff9936c732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:49.059572 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd3b688-4b07-434b-934e-ee6777f3923d service glance] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:49.060438 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd3b688-4b07-434b-934e-ee6777f3923d service glance] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:49.060438 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd3b688-4b07-434b-934e-ee6777f3923d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:49.067979 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e94909-0b7f-43d6-9177-e4e824b47cf1 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:49.069179 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd3b688-4b07-434b-934e-ee6777f3923d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:49.070664 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd3b688-4b07-434b-934e-ee6777f3923d service glance] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:49.073782 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6da773-06b7-4f28-a9a9-fd1b968532c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:49.074450 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6da773-06b7-4f28-a9a9-fd1b968532c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:49.074737 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6da773-06b7-4f28-a9a9-fd1b968532c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:49.075015 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6da773-06b7-4f28-a9a9-fd1b968532c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:49.075780 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6da773-06b7-4f28-a9a9-fd1b968532c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:49.077377 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6da773-06b7-4f28-a9a9-fd1b968532c0 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:49.080716 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1032/2082] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:04:48 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 22:04:49.089330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1051/2083] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 22:04:49.094012 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd3b688-4b07-434b-934e-ee6777f3923d service glance] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:49.105592 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd3b688-4b07-434b-934e-ee6777f3923d service glance] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:49.110197 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d0a79e-f859-4a9b-98af-3a6b37fc8b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:49.111075 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0a79e-f859-4a9b-98af-3a6b37fc8b1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:49.111406 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0a79e-f859-4a9b-98af-3a6b37fc8b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:49.111663 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0a79e-f859-4a9b-98af-3a6b37fc8b1e admin admin] PATH_INFO: `/v3/projects/a7e2e2c2d1854ab2ad4a415534ffd24f/users/f7ec31e2e7f94bb2885ff0af2a4a1395/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:49.128033 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d0a79e-f859-4a9b-98af-3a6b37fc8b1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7e2e2c2d1854ab2ad4a415534ffd24f, user_id=f7ec31e2e7f94bb2885ff0af2a4a1395, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:49.131292 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d0a79e-f859-4a9b-98af-3a6b37fc8b1e admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:49.144583 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1033/2084] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:04:49 2022] PUT /identity/v3/projects/a7e2e2c2d1854ab2ad4a415534ffd24f/users/f7ec31e2e7f94bb2885ff0af2a4a1395/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:04:49.155733 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d67de1-1432-4a8f-a96c-9bd2c5b5a207 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:49.156073 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d67de1-1432-4a8f-a96c-9bd2c5b5a207 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:49.156354 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d67de1-1432-4a8f-a96c-9bd2c5b5a207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:49.182991 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-c1d67de1-1432-4a8f-a96c-9bd2c5b5a207 None None] MFA Rules not processed for user `f7ec31e2e7f94bb2885ff0af2a4a1395`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:04:49.184250 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c1d67de1-1432-4a8f-a96c-9bd2c5b5a207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:49.253911 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1052/2085] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:04:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:04:49.258110 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d0a79e-f859-4a9b-98af-3a6b37fc8b1e admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:49.317283 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:04:49.333032 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d0a79e-f859-4a9b-98af-3a6b37fc8b1e admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:49.337475 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b2e1f8-594d-4168-b696-5f30c12ddf0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:49.338225 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b2e1f8-594d-4168-b696-5f30c12ddf0f admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:49.338579 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b2e1f8-594d-4168-b696-5f30c12ddf0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:49.338931 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b2e1f8-594d-4168-b696-5f30c12ddf0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:49.339876 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b2e1f8-594d-4168-b696-5f30c12ddf0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:49.341941 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b2e1f8-594d-4168-b696-5f30c12ddf0f admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:49.374569 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1034/2086] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:04:49 2022] POST /identity/v3/projects => generated 377 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:04:49.375760 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d67de1-1432-4a8f-a96c-9bd2c5b5a207 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:49.387840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d67de1-1432-4a8f-a96c-9bd2c5b5a207 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:49.460039 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2061d7-1f73-4b0e-a664-a46cc0430b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:49.460795 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2061d7-1f73-4b0e-a664-a46cc0430b1b admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:49.461145 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2061d7-1f73-4b0e-a664-a46cc0430b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:49.461496 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2061d7-1f73-4b0e-a664-a46cc0430b1b admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:49.462327 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2061d7-1f73-4b0e-a664-a46cc0430b1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:49.464346 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2061d7-1f73-4b0e-a664-a46cc0430b1b admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:49.487231 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1053/2087] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:04:49 2022] POST /identity/v3/users => generated 289 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:04:49.488938 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b2e1f8-594d-4168-b696-5f30c12ddf0f admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:49.500392 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b2e1f8-594d-4168-b696-5f30c12ddf0f admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:49.505445 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de94643f-1071-4057-82ff-c46491fb7f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:49.506103 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94643f-1071-4057-82ff-c46491fb7f6f admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:49.506386 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94643f-1071-4057-82ff-c46491fb7f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:49.506827 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94643f-1071-4057-82ff-c46491fb7f6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:49.507448 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de94643f-1071-4057-82ff-c46491fb7f6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:49.509076 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de94643f-1071-4057-82ff-c46491fb7f6f admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:49.520222 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1035/2088] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:04:49.524164 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2061d7-1f73-4b0e-a664-a46cc0430b1b admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:49.540592 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2061d7-1f73-4b0e-a664-a46cc0430b1b admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:49.545905 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8ee64f-1e7f-46b2-8629-f8eb49371e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:49.546577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ee64f-1e7f-46b2-8629-f8eb49371e0e admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:49.546920 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ee64f-1e7f-46b2-8629-f8eb49371e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:49.547163 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ee64f-1e7f-46b2-8629-f8eb49371e0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:49.548143 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8ee64f-1e7f-46b2-8629-f8eb49371e0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:49.551474 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8ee64f-1e7f-46b2-8629-f8eb49371e0e admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:49.568733 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1054/2089] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 22:04:49.572959 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de94643f-1071-4057-82ff-c46491fb7f6f admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:49.584496 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de94643f-1071-4057-82ff-c46491fb7f6f admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:49.592720 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b60dd37-c391-4b20-921c-541b43a43744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:49.593974 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60dd37-c391-4b20-921c-541b43a43744 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:49.594218 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60dd37-c391-4b20-921c-541b43a43744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:49.594633 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60dd37-c391-4b20-921c-541b43a43744 admin admin] PATH_INFO: `/v3/projects/d5512f18c4e14275bd4f0665debcea2b/users/a39961e867854cdfafddc0f78a4ea9e7/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:49.612486 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b60dd37-c391-4b20-921c-541b43a43744 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5512f18c4e14275bd4f0665debcea2b, user_id=a39961e867854cdfafddc0f78a4ea9e7, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:49.615184 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b60dd37-c391-4b20-921c-541b43a43744 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:49.630395 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1036/2090] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:04:49 2022] PUT /identity/v3/projects/d5512f18c4e14275bd4f0665debcea2b/users/a39961e867854cdfafddc0f78a4ea9e7/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:04:49.642262 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06be1bd-9b8c-48c6-849f-8de280f70ad4 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:49.642782 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06be1bd-9b8c-48c6-849f-8de280f70ad4 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:49.643141 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06be1bd-9b8c-48c6-849f-8de280f70ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:49.678078 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-f06be1bd-9b8c-48c6-849f-8de280f70ad4 None None] MFA Rules not processed for user `a39961e867854cdfafddc0f78a4ea9e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:04:49.679402 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-f06be1bd-9b8c-48c6-849f-8de280f70ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:49.702077 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:04:49.749755 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1055/2091] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:04:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:04:49.754284 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b60dd37-c391-4b20-921c-541b43a43744 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:49.817786 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b60dd37-c391-4b20-921c-541b43a43744 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:49.823308 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfc643a-02b2-4e7f-9099-849d2afaa9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:49.823994 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc643a-02b2-4e7f-9099-849d2afaa9bc admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:49.824283 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc643a-02b2-4e7f-9099-849d2afaa9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:49.824591 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc643a-02b2-4e7f-9099-849d2afaa9bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:49.825292 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfc643a-02b2-4e7f-9099-849d2afaa9bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:49.827183 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfc643a-02b2-4e7f-9099-849d2afaa9bc admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:49.853673 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1037/2092] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:04:49 2022] POST /identity/v3/projects => generated 377 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:04:49.857700 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06be1bd-9b8c-48c6-849f-8de280f70ad4 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:49.869377 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06be1bd-9b8c-48c6-849f-8de280f70ad4 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:49.928235 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39643d83-4659-42b0-ad7e-ab9ddd841fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:49.928914 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39643d83-4659-42b0-ad7e-ab9ddd841fb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:49.929201 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39643d83-4659-42b0-ad7e-ab9ddd841fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:49.929478 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39643d83-4659-42b0-ad7e-ab9ddd841fb2 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:49.930193 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39643d83-4659-42b0-ad7e-ab9ddd841fb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:49.933070 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39643d83-4659-42b0-ad7e-ab9ddd841fb2 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:49.953624 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1056/2093] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:04:49 2022] POST /identity/v3/users => generated 295 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:04:49.957548 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfc643a-02b2-4e7f-9099-849d2afaa9bc admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:49.968198 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfc643a-02b2-4e7f-9099-849d2afaa9bc admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:49.972880 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e718cdd0-d6ca-4edb-9aa9-4d08323307ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:49.973572 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e718cdd0-d6ca-4edb-9aa9-4d08323307ea admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:49.973857 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e718cdd0-d6ca-4edb-9aa9-4d08323307ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:49.974137 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e718cdd0-d6ca-4edb-9aa9-4d08323307ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:49.974875 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e718cdd0-d6ca-4edb-9aa9-4d08323307ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:49.976822 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e718cdd0-d6ca-4edb-9aa9-4d08323307ea admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:49.987786 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1038/2094] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:04:49.992122 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39643d83-4659-42b0-ad7e-ab9ddd841fb2 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:50.003740 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39643d83-4659-42b0-ad7e-ab9ddd841fb2 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:50.009857 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af08ec45-aa94-435d-b662-53d9f0bb1995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:50.011269 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08ec45-aa94-435d-b662-53d9f0bb1995 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:50.011719 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08ec45-aa94-435d-b662-53d9f0bb1995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:50.012130 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08ec45-aa94-435d-b662-53d9f0bb1995 admin admin] PATH_INFO: `/v3/projects/5d578d9e1a99467ba580b6ed0dc1287f/users/188e0d449da24111b36d2b482106ee11/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:50.034410 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af08ec45-aa94-435d-b662-53d9f0bb1995 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d578d9e1a99467ba580b6ed0dc1287f, user_id=188e0d449da24111b36d2b482106ee11, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:50.037333 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af08ec45-aa94-435d-b662-53d9f0bb1995 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:50.055259 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1057/2095] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:04:49 2022] PUT /identity/v3/projects/5d578d9e1a99467ba580b6ed0dc1287f/users/188e0d449da24111b36d2b482106ee11/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:04:50.059624 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e718cdd0-d6ca-4edb-9aa9-4d08323307ea admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:50.070663 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e718cdd0-d6ca-4edb-9aa9-4d08323307ea admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:50.147031 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814ddca0-2e02-4be5-b506-dff59d356ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:50.148372 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ddca0-2e02-4be5-b506-dff59d356ae9 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:50.148726 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ddca0-2e02-4be5-b506-dff59d356ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:50.149235 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ddca0-2e02-4be5-b506-dff59d356ae9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:50.152233 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814ddca0-2e02-4be5-b506-dff59d356ae9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:50.154126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814ddca0-2e02-4be5-b506-dff59d356ae9 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:50.173163 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1039/2096] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:04:50 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:04:50.174321 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af08ec45-aa94-435d-b662-53d9f0bb1995 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:50.192997 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af08ec45-aa94-435d-b662-53d9f0bb1995 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:50.210405 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf221b8f-1130-4675-a118-666a6a0a71c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:50.211423 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf221b8f-1130-4675-a118-666a6a0a71c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:50.211916 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf221b8f-1130-4675-a118-666a6a0a71c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:50.212198 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf221b8f-1130-4675-a118-666a6a0a71c6 admin admin] PATH_INFO: `/v3/projects/5d578d9e1a99467ba580b6ed0dc1287f/users/188e0d449da24111b36d2b482106ee11/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:50.217318 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf221b8f-1130-4675-a118-666a6a0a71c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d578d9e1a99467ba580b6ed0dc1287f, user_id=188e0d449da24111b36d2b482106ee11, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:50.220054 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf221b8f-1130-4675-a118-666a6a0a71c6 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:50.254813 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1058/2097] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:04:50 2022] PUT /identity/v3/projects/5d578d9e1a99467ba580b6ed0dc1287f/users/188e0d449da24111b36d2b482106ee11/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:04:50.265545 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7eed7e-ae46-43cb-8b18-d58ed2638c46 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:50.265885 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7eed7e-ae46-43cb-8b18-d58ed2638c46 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:50.266117 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7eed7e-ae46-43cb-8b18-d58ed2638c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:50.295080 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:04:50.316004 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-cd7eed7e-ae46-43cb-8b18-d58ed2638c46 None None] MFA Rules not processed for user `188e0d449da24111b36d2b482106ee11`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:04:50.317249 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-cd7eed7e-ae46-43cb-8b18-d58ed2638c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:50.472473 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1040/2098] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:04:50 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:04:50.481205 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf221b8f-1130-4675-a118-666a6a0a71c6 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:50.482973 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-bf221b8f-1130-4675-a118-666a6a0a71c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:50.592319 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf221b8f-1130-4675-a118-666a6a0a71c6 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:50.597927 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b20796-8b82-4f51-8c2c-a621f1939296 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:50.599606 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b20796-8b82-4f51-8c2c-a621f1939296 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:50.602058 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b20796-8b82-4f51-8c2c-a621f1939296 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:50.602915 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b20796-8b82-4f51-8c2c-a621f1939296 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:50.656263 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:04:50.689725 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b20796-8b82-4f51-8c2c-a621f1939296 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:50.691471 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b20796-8b82-4f51-8c2c-a621f1939296 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:50.702094 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1059/2099] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:04:50 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:04:53.380234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7eed7e-ae46-43cb-8b18-d58ed2638c46 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:53.381767 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-cd7eed7e-ae46-43cb-8b18-d58ed2638c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:53.394016 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7eed7e-ae46-43cb-8b18-d58ed2638c46 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:53.397604 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e71064-cdd3-4ed0-97b7-bf967c251e46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:53.398250 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e71064-cdd3-4ed0-97b7-bf967c251e46 service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:53.398530 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e71064-cdd3-4ed0-97b7-bf967c251e46 service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:53.398802 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e71064-cdd3-4ed0-97b7-bf967c251e46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:53.465765 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e71064-cdd3-4ed0-97b7-bf967c251e46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:53.467527 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e71064-cdd3-4ed0-97b7-bf967c251e46 service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:53.477865 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1041/2100] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:04:53 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:04:54.124469 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b20796-8b82-4f51-8c2c-a621f1939296 service cinder] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:54.132856 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b20796-8b82-4f51-8c2c-a621f1939296 service cinder] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:54.137100 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a7683d-d83a-42f6-a0d8-293fa732829c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:54.137731 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7683d-d83a-42f6-a0d8-293fa732829c service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:54.138016 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7683d-d83a-42f6-a0d8-293fa732829c service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:54.138290 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7683d-d83a-42f6-a0d8-293fa732829c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:54.152048 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a7683d-d83a-42f6-a0d8-293fa732829c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:54.153535 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a7683d-d83a-42f6-a0d8-293fa732829c service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:54.164158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1060/2101] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:04:54 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:04:55.808515 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c08d7d-2eff-446c-a30c-292e252f6a00 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:55.809091 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c08d7d-2eff-446c-a30c-292e252f6a00 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:55.809091 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c08d7d-2eff-446c-a30c-292e252f6a00 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:55.810478 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1042/2102] 10.222.0.13 () {54 vars in 921 bytes} [Tue Jun 7 22:04:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 22:04:55.833886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e195ad0-bd7a-472f-a70b-389bd539fea0 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:55.834372 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e195ad0-bd7a-472f-a70b-389bd539fea0 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:55.834491 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e195ad0-bd7a-472f-a70b-389bd539fea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:55.854238 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-8e195ad0-bd7a-472f-a70b-389bd539fea0 None None] MFA Rules not processed for user `02941c6ad2d54e48b1114d9e030c66c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:04:55.855534 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-8e195ad0-bd7a-472f-a70b-389bd539fea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:55.871218 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c08d7d-2eff-446c-a30c-292e252f6a00 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:55.873153 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-51c08d7d-2eff-446c-a30c-292e252f6a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:55.924722 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1061/2103] 10.222.0.13 () {58 vars in 1018 bytes} [Tue Jun 7 22:04:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:04:55.936195 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e195ad0-bd7a-472f-a70b-389bd539fea0 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:55.937539 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-8e195ad0-bd7a-472f-a70b-389bd539fea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:55.950501 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e195ad0-bd7a-472f-a70b-389bd539fea0 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:55.954475 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c08d7d-2eff-446c-a30c-292e252f6a00 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:55.955691 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fe4bd1-8e6b-4f8f-a281-d6f9b7ccd4e6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02941c6ad2d54e48b1114d9e030c66c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:55.956359 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe4bd1-8e6b-4f8f-a281-d6f9b7ccd4e6 service placement] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:55.956640 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe4bd1-8e6b-4f8f-a281-d6f9b7ccd4e6 service placement] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:55.956924 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe4bd1-8e6b-4f8f-a281-d6f9b7ccd4e6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:55.959239 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d680ad-e18b-4e8b-baca-4dc6b23b6f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:55.960143 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d680ad-e18b-4e8b-baca-4dc6b23b6f78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:55.960429 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d680ad-e18b-4e8b-baca-4dc6b23b6f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:55.960714 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d680ad-e18b-4e8b-baca-4dc6b23b6f78 admin admin] PATH_INFO: `/v3/users/1378a898f9534635857e5958d32f00c5` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:55.961994 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d680ad-e18b-4e8b-baca-4dc6b23b6f78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1378a898f9534635857e5958d32f00c5)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:55.965102 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d680ad-e18b-4e8b-baca-4dc6b23b6f78 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:55.965767 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fe4bd1-8e6b-4f8f-a281-d6f9b7ccd4e6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:55.967387 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fe4bd1-8e6b-4f8f-a281-d6f9b7ccd4e6 service placement] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:55.979687 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1062/2104] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:04:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 22:04:56.020659 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:04:56.030791 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c1d680ad-e18b-4e8b-baca-4dc6b23b6f78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1378a898f9534635857e5958d32f00c5'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:56.031496 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c1d680ad-e18b-4e8b-baca-4dc6b23b6f78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1378a898f9534635857e5958d32f00c5'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:56.038964 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c1d680ad-e18b-4e8b-baca-4dc6b23b6f78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1378a898f9534635857e5958d32f00c5'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:56.052854 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c1d680ad-e18b-4e8b-baca-4dc6b23b6f78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1378a898f9534635857e5958d32f00c5'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:56.060504 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c1d680ad-e18b-4e8b-baca-4dc6b23b6f78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1378a898f9534635857e5958d32f00c5'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:56.066817 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1043/2105] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:04:55 2022] DELETE /identity/v3/users/1378a898f9534635857e5958d32f00c5 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:04:56.327535 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fe4bd1-8e6b-4f8f-a281-d6f9b7ccd4e6 service placement] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:56.338212 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fe4bd1-8e6b-4f8f-a281-d6f9b7ccd4e6 service placement] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:56.397473 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb0d504-9055-4e0d-bab3-0d4eecbd486b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:56.398327 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb0d504-9055-4e0d-bab3-0d4eecbd486b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:56.398605 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb0d504-9055-4e0d-bab3-0d4eecbd486b admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:56.401258 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb0d504-9055-4e0d-bab3-0d4eecbd486b admin admin] PATH_INFO: `/v3/projects/7e8d5b4fdf9a48a296359e167017123f` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:56.401258 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb0d504-9055-4e0d-bab3-0d4eecbd486b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e8d5b4fdf9a48a296359e167017123f)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:56.402575 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb0d504-9055-4e0d-bab3-0d4eecbd486b admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:56.448210 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:04:56.454888 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-2fb0d504-9055-4e0d-bab3-0d4eecbd486b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e8d5b4fdf9a48a296359e167017123f', 'request_id': 'req-2fb0d504-9055-4e0d-bab3-0d4eecbd486b'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:56.463245 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-2fb0d504-9055-4e0d-bab3-0d4eecbd486b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e8d5b4fdf9a48a296359e167017123f', 'request_id': 'req-2fb0d504-9055-4e0d-bab3-0d4eecbd486b'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:56.469555 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-2fb0d504-9055-4e0d-bab3-0d4eecbd486b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e8d5b4fdf9a48a296359e167017123f', 'request_id': 'req-2fb0d504-9055-4e0d-bab3-0d4eecbd486b'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:56.485148 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-2fb0d504-9055-4e0d-bab3-0d4eecbd486b admin admin] The token cache is being invalidate because project 7e8d5b4fdf9a48a296359e167017123f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:04:56.485434 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-2fb0d504-9055-4e0d-bab3-0d4eecbd486b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:56.486082 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-2fb0d504-9055-4e0d-bab3-0d4eecbd486b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:56.487697 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1063/2106] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:04:56 2022] DELETE /identity/v3/projects/7e8d5b4fdf9a48a296359e167017123f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:04:56.493519 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a0c63-efd4-4b70-959e-66cd19c23487 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:56.494325 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a0c63-efd4-4b70-959e-66cd19c23487 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:56.494426 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a0c63-efd4-4b70-959e-66cd19c23487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:56.513811 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-4d7a0c63-efd4-4b70-959e-66cd19c23487 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:04:56.515018 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-4d7a0c63-efd4-4b70-959e-66cd19c23487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:56.579471 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1044/2107] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:04:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:04:56.587798 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f71ef-fa51-4464-bff1-bdc6b968b234 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:56.588094 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f71ef-fa51-4464-bff1-bdc6b968b234 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:56.588368 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f71ef-fa51-4464-bff1-bdc6b968b234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:56.607471 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-a85f71ef-fa51-4464-bff1-bdc6b968b234 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:04:56.608718 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a85f71ef-fa51-4464-bff1-bdc6b968b234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:56.616644 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1064/2108] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:04:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:04:56.622276 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7a0c63-efd4-4b70-959e-66cd19c23487 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:56.632782 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7a0c63-efd4-4b70-959e-66cd19c23487 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:56.640195 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c77c9-5e9d-42e3-b31a-4158822a4709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:56.641023 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c77c9-5e9d-42e3-b31a-4158822a4709 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:56.641387 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c77c9-5e9d-42e3-b31a-4158822a4709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:56.641730 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c77c9-5e9d-42e3-b31a-4158822a4709 admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:56.642544 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c77c9-5e9d-42e3-b31a-4158822a4709 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:04:56.643072 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c77c9-5e9d-42e3-b31a-4158822a4709 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:56.645520 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c77c9-5e9d-42e3-b31a-4158822a4709 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:56.664079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1045/2109] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:04:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:04:56.668868 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f71ef-fa51-4464-bff1-bdc6b968b234 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:56.678965 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f71ef-fa51-4464-bff1-bdc6b968b234 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:56.684947 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea34445-66f2-45ce-8d2b-d2d851238991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:56.685724 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea34445-66f2-45ce-8d2b-d2d851238991 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:56.686068 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea34445-66f2-45ce-8d2b-d2d851238991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:56.686569 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea34445-66f2-45ce-8d2b-d2d851238991 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:56.690267 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea34445-66f2-45ce-8d2b-d2d851238991 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:56.691761 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea34445-66f2-45ce-8d2b-d2d851238991 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:56.734774 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1065/2110] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:04:56 2022] POST /identity/v3/projects => generated 389 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:04:56.735942 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c77c9-5e9d-42e3-b31a-4158822a4709 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:56.749155 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c77c9-5e9d-42e3-b31a-4158822a4709 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:56.822761 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3479b9-c10f-4c86-9d79-e3ce853726d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:56.823511 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3479b9-c10f-4c86-9d79-e3ce853726d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:56.823752 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3479b9-c10f-4c86-9d79-e3ce853726d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:56.824035 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3479b9-c10f-4c86-9d79-e3ce853726d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:56.824773 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3479b9-c10f-4c86-9d79-e3ce853726d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:56.828593 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3479b9-c10f-4c86-9d79-e3ce853726d7 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:56.857002 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1046/2111] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:04:56 2022] POST /identity/v3/users => generated 295 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:04:56.858371 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea34445-66f2-45ce-8d2b-d2d851238991 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:56.870220 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea34445-66f2-45ce-8d2b-d2d851238991 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:56.875778 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f26c63-2b49-46be-a3d0-382de1691d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:56.876451 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f26c63-2b49-46be-a3d0-382de1691d80 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:56.876730 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f26c63-2b49-46be-a3d0-382de1691d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:56.876998 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f26c63-2b49-46be-a3d0-382de1691d80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:56.877729 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f26c63-2b49-46be-a3d0-382de1691d80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:56.879580 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f26c63-2b49-46be-a3d0-382de1691d80 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:56.892615 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1066/2112] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:04:56.897125 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3479b9-c10f-4c86-9d79-e3ce853726d7 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:56.911482 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3479b9-c10f-4c86-9d79-e3ce853726d7 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:56.917007 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ea51ae-24e1-420f-9f6f-337bd0cd0753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:56.917630 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea51ae-24e1-420f-9f6f-337bd0cd0753 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:56.917914 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea51ae-24e1-420f-9f6f-337bd0cd0753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:56.918290 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea51ae-24e1-420f-9f6f-337bd0cd0753 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:56.919260 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ea51ae-24e1-420f-9f6f-337bd0cd0753 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:56.921449 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ea51ae-24e1-420f-9f6f-337bd0cd0753 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:56.934398 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1047/2113] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:04:56.938485 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f26c63-2b49-46be-a3d0-382de1691d80 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:56.949592 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f26c63-2b49-46be-a3d0-382de1691d80 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:56.955456 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd4226-a82c-4e39-acbb-d5f40dc4a582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:56.956781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd4226-a82c-4e39-acbb-d5f40dc4a582 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:56.957307 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd4226-a82c-4e39-acbb-d5f40dc4a582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:56.957829 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd4226-a82c-4e39-acbb-d5f40dc4a582 admin admin] PATH_INFO: `/v3/projects/9f0ee4be50354a20baf5af85aeb4c3b2/users/de021dc74a4446d8b6a63883fd56a7c4/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:56.985549 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cd4226-a82c-4e39-acbb-d5f40dc4a582 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f0ee4be50354a20baf5af85aeb4c3b2, user_id=de021dc74a4446d8b6a63883fd56a7c4, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:56.988389 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cd4226-a82c-4e39-acbb-d5f40dc4a582 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:57.006350 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1067/2114] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:04:56 2022] PUT /identity/v3/projects/9f0ee4be50354a20baf5af85aeb4c3b2/users/de021dc74a4446d8b6a63883fd56a7c4/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:04:57.011465 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d488f-5abc-47d6-a209-37fb317ea9f3 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:57.011877 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d488f-5abc-47d6-a209-37fb317ea9f3 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:57.012358 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d488f-5abc-47d6-a209-37fb317ea9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:57.038906 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-524d488f-5abc-47d6-a209-37fb317ea9f3 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:04:57.040867 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-524d488f-5abc-47d6-a209-37fb317ea9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:57.062868 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:04:57.120172 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1048/2115] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:04:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:04:57.129999 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd4226-a82c-4e39-acbb-d5f40dc4a582 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:57.134439 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-49cd4226-a82c-4e39-acbb-d5f40dc4a582 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:57.194805 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd4226-a82c-4e39-acbb-d5f40dc4a582 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:57.199522 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffab4d3-79b3-4a29-961c-98724ead9ed1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:57.200572 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffab4d3-79b3-4a29-961c-98724ead9ed1 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:57.200812 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffab4d3-79b3-4a29-961c-98724ead9ed1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:57.201088 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffab4d3-79b3-4a29-961c-98724ead9ed1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:57.211172 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffab4d3-79b3-4a29-961c-98724ead9ed1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:57.213197 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffab4d3-79b3-4a29-961c-98724ead9ed1 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:57.222939 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1068/2116] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:04:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:04:58.195820 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d488f-5abc-47d6-a209-37fb317ea9f3 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:58.197983 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-524d488f-5abc-47d6-a209-37fb317ea9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:58.210073 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d488f-5abc-47d6-a209-37fb317ea9f3 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:58.216184 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9af13ee-7833-4bac-9012-ae13790a0812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:58.216583 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af13ee-7833-4bac-9012-ae13790a0812 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:58.216870 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af13ee-7833-4bac-9012-ae13790a0812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:58.217146 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af13ee-7833-4bac-9012-ae13790a0812 admin admin] PATH_INFO: `/v3/users/f7ec31e2e7f94bb2885ff0af2a4a1395` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:58.218809 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9af13ee-7833-4bac-9012-ae13790a0812 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7ec31e2e7f94bb2885ff0af2a4a1395)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:58.220417 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9af13ee-7833-4bac-9012-ae13790a0812 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:58.282810 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c9af13ee-7833-4bac-9012-ae13790a0812 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7ec31e2e7f94bb2885ff0af2a4a1395'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:58.283650 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c9af13ee-7833-4bac-9012-ae13790a0812 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7ec31e2e7f94bb2885ff0af2a4a1395'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:58.294243 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c9af13ee-7833-4bac-9012-ae13790a0812 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7ec31e2e7f94bb2885ff0af2a4a1395'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:58.315821 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c9af13ee-7833-4bac-9012-ae13790a0812 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7ec31e2e7f94bb2885ff0af2a4a1395'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:58.324381 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c9af13ee-7833-4bac-9012-ae13790a0812 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7ec31e2e7f94bb2885ff0af2a4a1395'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:58.331497 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1049/2117] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:04:58 2022] DELETE /identity/v3/users/f7ec31e2e7f94bb2885ff0af2a4a1395 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:04:58.336456 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272be5cb-8739-4b0a-875a-64ce23d328e4 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:58.337131 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272be5cb-8739-4b0a-875a-64ce23d328e4 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:58.337131 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272be5cb-8739-4b0a-875a-64ce23d328e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:58.357564 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-272be5cb-8739-4b0a-875a-64ce23d328e4 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:04:58.358920 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-272be5cb-8739-4b0a-875a-64ce23d328e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:58.430912 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1069/2118] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:04:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:04:58.441475 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9af13ee-7833-4bac-9012-ae13790a0812 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:58.443024 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-c9af13ee-7833-4bac-9012-ae13790a0812 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:58.522985 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9af13ee-7833-4bac-9012-ae13790a0812 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:58.528598 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f113faed-d1ef-48b0-9d14-ab42c3c56cb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:58.529961 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113faed-d1ef-48b0-9d14-ab42c3c56cb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:58.530374 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113faed-d1ef-48b0-9d14-ab42c3c56cb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:58.531063 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113faed-d1ef-48b0-9d14-ab42c3c56cb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:58.546122 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f113faed-d1ef-48b0-9d14-ab42c3c56cb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:58.549002 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f113faed-d1ef-48b0-9d14-ab42c3c56cb0 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:58.568782 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1050/2119] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:04:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:04:59.124874 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272be5cb-8739-4b0a-875a-64ce23d328e4 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:59.136122 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-272be5cb-8739-4b0a-875a-64ce23d328e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:04:59.181999 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272be5cb-8739-4b0a-875a-64ce23d328e4 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:59.189183 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0966e439-628e-4339-9e32-1b4d6a04cfa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:59.192279 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966e439-628e-4339-9e32-1b4d6a04cfa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:59.192813 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966e439-628e-4339-9e32-1b4d6a04cfa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:59.193901 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966e439-628e-4339-9e32-1b4d6a04cfa0 admin admin] PATH_INFO: `/v3/projects/a7e2e2c2d1854ab2ad4a415534ffd24f` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:59.196344 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0966e439-628e-4339-9e32-1b4d6a04cfa0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7e2e2c2d1854ab2ad4a415534ffd24f)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:59.198669 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0966e439-628e-4339-9e32-1b4d6a04cfa0 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:59.203008 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:04:59.322357 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-0966e439-628e-4339-9e32-1b4d6a04cfa0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7e2e2c2d1854ab2ad4a415534ffd24f', 'request_id': 'req-0966e439-628e-4339-9e32-1b4d6a04cfa0'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:59.336108 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-0966e439-628e-4339-9e32-1b4d6a04cfa0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7e2e2c2d1854ab2ad4a415534ffd24f', 'request_id': 'req-0966e439-628e-4339-9e32-1b4d6a04cfa0'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:59.346636 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-0966e439-628e-4339-9e32-1b4d6a04cfa0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7e2e2c2d1854ab2ad4a415534ffd24f', 'request_id': 'req-0966e439-628e-4339-9e32-1b4d6a04cfa0'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:59.372915 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-0966e439-628e-4339-9e32-1b4d6a04cfa0 admin admin] The token cache is being invalidate because project a7e2e2c2d1854ab2ad4a415534ffd24f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:04:59.373786 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-0966e439-628e-4339-9e32-1b4d6a04cfa0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:59.374651 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-0966e439-628e-4339-9e32-1b4d6a04cfa0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:59.376392 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1070/2120] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:04:59 2022] DELETE /identity/v3/projects/a7e2e2c2d1854ab2ad4a415534ffd24f => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:04:59.380865 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f113faed-d1ef-48b0-9d14-ab42c3c56cb0 service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:04:59.393062 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f113faed-d1ef-48b0-9d14-ab42c3c56cb0 service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:04:59.471376 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:04:59.566772 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0fc119-86dc-4df8-801c-cdc08fff5593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:04:59.568031 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0fc119-86dc-4df8-801c-cdc08fff5593 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:04:59.568395 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0fc119-86dc-4df8-801c-cdc08fff5593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:04:59.568772 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0fc119-86dc-4df8-801c-cdc08fff5593 admin admin] PATH_INFO: `/v3/users/a39961e867854cdfafddc0f78a4ea9e7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:04:59.570330 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0fc119-86dc-4df8-801c-cdc08fff5593 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a39961e867854cdfafddc0f78a4ea9e7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:04:59.572715 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0fc119-86dc-4df8-801c-cdc08fff5593 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:04:59.683608 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-7f0fc119-86dc-4df8-801c-cdc08fff5593 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a39961e867854cdfafddc0f78a4ea9e7'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:59.684619 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-7f0fc119-86dc-4df8-801c-cdc08fff5593 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a39961e867854cdfafddc0f78a4ea9e7'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:59.701711 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-7f0fc119-86dc-4df8-801c-cdc08fff5593 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a39961e867854cdfafddc0f78a4ea9e7'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:59.726768 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-7f0fc119-86dc-4df8-801c-cdc08fff5593 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a39961e867854cdfafddc0f78a4ea9e7'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:59.736319 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-7f0fc119-86dc-4df8-801c-cdc08fff5593 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a39961e867854cdfafddc0f78a4ea9e7'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:04:59.759455 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1051/2121] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:04:59 2022] DELETE /identity/v3/users/a39961e867854cdfafddc0f78a4ea9e7 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:05:00.530656 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0966e439-628e-4339-9e32-1b4d6a04cfa0 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:00.532285 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-0966e439-628e-4339-9e32-1b4d6a04cfa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:05:00.553693 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0966e439-628e-4339-9e32-1b4d6a04cfa0 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:00.655457 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be255ae-391b-4cae-b034-f41512362444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:00.656540 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be255ae-391b-4cae-b034-f41512362444 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:00.656821 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be255ae-391b-4cae-b034-f41512362444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:00.657719 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be255ae-391b-4cae-b034-f41512362444 admin admin] PATH_INFO: `/v3/projects/d5512f18c4e14275bd4f0665debcea2b` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:00.659224 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be255ae-391b-4cae-b034-f41512362444 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5512f18c4e14275bd4f0665debcea2b)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:00.661388 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be255ae-391b-4cae-b034-f41512362444 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:00.733446 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8be255ae-391b-4cae-b034-f41512362444 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5512f18c4e14275bd4f0665debcea2b', 'request_id': 'req-8be255ae-391b-4cae-b034-f41512362444'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:05:00.742119 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8be255ae-391b-4cae-b034-f41512362444 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5512f18c4e14275bd4f0665debcea2b', 'request_id': 'req-8be255ae-391b-4cae-b034-f41512362444'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:05:00.752277 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8be255ae-391b-4cae-b034-f41512362444 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5512f18c4e14275bd4f0665debcea2b', 'request_id': 'req-8be255ae-391b-4cae-b034-f41512362444'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:05:00.775085 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8be255ae-391b-4cae-b034-f41512362444 admin admin] The token cache is being invalidate because project d5512f18c4e14275bd4f0665debcea2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:05:00.775392 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8be255ae-391b-4cae-b034-f41512362444 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:05:00.776792 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8be255ae-391b-4cae-b034-f41512362444 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:05:00.778500 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1071/2122] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:05:00 2022] DELETE /identity/v3/projects/d5512f18c4e14275bd4f0665debcea2b => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:05:00.782498 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0fc119-86dc-4df8-801c-cdc08fff5593 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:00.783951 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-7f0fc119-86dc-4df8-801c-cdc08fff5593 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:05:00.797215 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0fc119-86dc-4df8-801c-cdc08fff5593 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:00.895723 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0300ff4f-fffb-4962-a268-c9f6f977ad65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:00.897274 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300ff4f-fffb-4962-a268-c9f6f977ad65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:00.897274 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300ff4f-fffb-4962-a268-c9f6f977ad65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:00.897455 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300ff4f-fffb-4962-a268-c9f6f977ad65 admin admin] PATH_INFO: `/v3/users/188e0d449da24111b36d2b482106ee11` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:00.900002 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0300ff4f-fffb-4962-a268-c9f6f977ad65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=188e0d449da24111b36d2b482106ee11)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:00.902119 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0300ff4f-fffb-4962-a268-c9f6f977ad65 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:00.985152 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-0300ff4f-fffb-4962-a268-c9f6f977ad65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '188e0d449da24111b36d2b482106ee11'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:05:00.985948 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-0300ff4f-fffb-4962-a268-c9f6f977ad65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '188e0d449da24111b36d2b482106ee11'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:05:01.007542 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-0300ff4f-fffb-4962-a268-c9f6f977ad65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '188e0d449da24111b36d2b482106ee11'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:05:01.010544 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:05:01.038866 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-0300ff4f-fffb-4962-a268-c9f6f977ad65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '188e0d449da24111b36d2b482106ee11'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:05:01.055886 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-0300ff4f-fffb-4962-a268-c9f6f977ad65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '188e0d449da24111b36d2b482106ee11'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:05:01.072660 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1052/2123] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:05:00 2022] DELETE /identity/v3/users/188e0d449da24111b36d2b482106ee11 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:05:01.540291 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be255ae-391b-4cae-b034-f41512362444 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:01.544503 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-8be255ae-391b-4cae-b034-f41512362444 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:05:01.574920 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be255ae-391b-4cae-b034-f41512362444 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:01.621307 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:05:01.692246 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360afc40-9a48-48de-b4e6-dfa936301313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:01.693411 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360afc40-9a48-48de-b4e6-dfa936301313 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:01.694293 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360afc40-9a48-48de-b4e6-dfa936301313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:01.694830 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360afc40-9a48-48de-b4e6-dfa936301313 admin admin] PATH_INFO: `/v3/projects/5d578d9e1a99467ba580b6ed0dc1287f` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:01.696821 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360afc40-9a48-48de-b4e6-dfa936301313 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d578d9e1a99467ba580b6ed0dc1287f)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:01.698924 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360afc40-9a48-48de-b4e6-dfa936301313 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:01.807565 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-360afc40-9a48-48de-b4e6-dfa936301313 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d578d9e1a99467ba580b6ed0dc1287f', 'request_id': 'req-360afc40-9a48-48de-b4e6-dfa936301313'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:05:01.819399 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-360afc40-9a48-48de-b4e6-dfa936301313 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d578d9e1a99467ba580b6ed0dc1287f', 'request_id': 'req-360afc40-9a48-48de-b4e6-dfa936301313'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:05:01.836110 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-360afc40-9a48-48de-b4e6-dfa936301313 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d578d9e1a99467ba580b6ed0dc1287f', 'request_id': 'req-360afc40-9a48-48de-b4e6-dfa936301313'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:05:01.857422 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-360afc40-9a48-48de-b4e6-dfa936301313 admin admin] The token cache is being invalidate because project 5d578d9e1a99467ba580b6ed0dc1287f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:05:01.857719 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-360afc40-9a48-48de-b4e6-dfa936301313 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:05:01.858431 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-360afc40-9a48-48de-b4e6-dfa936301313 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:05:01.863287 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1072/2124] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:05:01 2022] DELETE /identity/v3/projects/5d578d9e1a99467ba580b6ed0dc1287f => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:05:01.865745 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef2c86-0e38-47c2-bafa-f087da2db1ba None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:01.866832 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef2c86-0e38-47c2-bafa-f087da2db1ba None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:01.867299 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef2c86-0e38-47c2-bafa-f087da2db1ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:01.888548 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-ccef2c86-0e38-47c2-bafa-f087da2db1ba None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:05:01.890689 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-ccef2c86-0e38-47c2-bafa-f087da2db1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:05:01.987804 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1053/2125] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:05:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:05:01.999279 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a90b6-5289-455f-b014-977bd0106c39 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:02.002837 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a90b6-5289-455f-b014-977bd0106c39 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:02.003137 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a90b6-5289-455f-b014-977bd0106c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:02.043372 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-788a90b6-5289-455f-b014-977bd0106c39 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:05:02.045113 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-788a90b6-5289-455f-b014-977bd0106c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:05:02.070909 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1073/2126] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:05:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:05:02.072427 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccef2c86-0e38-47c2-bafa-f087da2db1ba None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:02.098682 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccef2c86-0e38-47c2-bafa-f087da2db1ba None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:02.098682 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322bc2e9-fdfe-4e0d-bbf0-e050fbe6f04d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:02.098682 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322bc2e9-fdfe-4e0d-bbf0-e050fbe6f04d admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:02.098682 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322bc2e9-fdfe-4e0d-bbf0-e050fbe6f04d admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:02.099212 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322bc2e9-fdfe-4e0d-bbf0-e050fbe6f04d admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:02.099212 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322bc2e9-fdfe-4e0d-bbf0-e050fbe6f04d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:05:02.099212 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322bc2e9-fdfe-4e0d-bbf0-e050fbe6f04d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:02.100802 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322bc2e9-fdfe-4e0d-bbf0-e050fbe6f04d admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:02.119846 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1054/2127] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:05:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:05:02.124111 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788a90b6-5289-455f-b014-977bd0106c39 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:02.134744 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788a90b6-5289-455f-b014-977bd0106c39 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:02.141393 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e269e7b5-f534-4b47-808a-311abffbdab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:02.142068 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e269e7b5-f534-4b47-808a-311abffbdab8 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:02.142719 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e269e7b5-f534-4b47-808a-311abffbdab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:02.143002 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e269e7b5-f534-4b47-808a-311abffbdab8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:02.144096 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e269e7b5-f534-4b47-808a-311abffbdab8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:02.146264 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e269e7b5-f534-4b47-808a-311abffbdab8 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:02.194606 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1074/2128] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:05:02 2022] POST /identity/v3/projects => generated 373 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:05:02.199191 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322bc2e9-fdfe-4e0d-bbf0-e050fbe6f04d admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:02.211695 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322bc2e9-fdfe-4e0d-bbf0-e050fbe6f04d admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:02.322968 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0e78f-248b-456e-be2d-da7cb04c7a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:02.323611 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0e78f-248b-456e-be2d-da7cb04c7a37 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:02.323899 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0e78f-248b-456e-be2d-da7cb04c7a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:02.324184 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0e78f-248b-456e-be2d-da7cb04c7a37 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:02.324896 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d0e78f-248b-456e-be2d-da7cb04c7a37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:02.328191 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d0e78f-248b-456e-be2d-da7cb04c7a37 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:02.366648 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1055/2129] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:05:02 2022] POST /identity/v3/users => generated 287 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:05:02.371771 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e269e7b5-f534-4b47-808a-311abffbdab8 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:02.386649 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e269e7b5-f534-4b47-808a-311abffbdab8 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:02.393355 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43811696-e2c6-4e20-bf81-55dd19da1a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:02.394125 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43811696-e2c6-4e20-bf81-55dd19da1a9c admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:02.394612 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43811696-e2c6-4e20-bf81-55dd19da1a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:02.394941 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43811696-e2c6-4e20-bf81-55dd19da1a9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:02.395925 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43811696-e2c6-4e20-bf81-55dd19da1a9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:02.399792 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43811696-e2c6-4e20-bf81-55dd19da1a9c admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:02.411635 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1075/2130] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:05:02 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:05:02.415971 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0e78f-248b-456e-be2d-da7cb04c7a37 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:02.431836 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0e78f-248b-456e-be2d-da7cb04c7a37 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:02.437209 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0038a0be-a60e-4275-b8c8-8c399f96b665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:02.438041 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038a0be-a60e-4275-b8c8-8c399f96b665 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:02.438330 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038a0be-a60e-4275-b8c8-8c399f96b665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:02.438609 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038a0be-a60e-4275-b8c8-8c399f96b665 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:02.439399 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0038a0be-a60e-4275-b8c8-8c399f96b665 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:02.440984 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0038a0be-a60e-4275-b8c8-8c399f96b665 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:02.455652 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1056/2131] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:05:02 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 22:05:02.460329 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43811696-e2c6-4e20-bf81-55dd19da1a9c admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:02.479487 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43811696-e2c6-4e20-bf81-55dd19da1a9c admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:02.504102 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c09ef91-39d8-42ba-bc0f-4521c39ed2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:02.505073 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09ef91-39d8-42ba-bc0f-4521c39ed2c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:02.505369 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09ef91-39d8-42ba-bc0f-4521c39ed2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:02.505638 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09ef91-39d8-42ba-bc0f-4521c39ed2c8 admin admin] PATH_INFO: `/v3/projects/bc558dd3848d4d3d950d7cd408e19741/users/7ce9ec4aea9449d3890f6dbb93f97968/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:02.529115 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c09ef91-39d8-42ba-bc0f-4521c39ed2c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc558dd3848d4d3d950d7cd408e19741, user_id=7ce9ec4aea9449d3890f6dbb93f97968, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:02.531764 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c09ef91-39d8-42ba-bc0f-4521c39ed2c8 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:02.547773 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1076/2132] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:05:02 2022] PUT /identity/v3/projects/bc558dd3848d4d3d950d7cd408e19741/users/7ce9ec4aea9449d3890f6dbb93f97968/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:05:02.553304 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b98c2-8a2b-4273-8f4d-9e52129dc686 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:02.553600 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b98c2-8a2b-4273-8f4d-9e52129dc686 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:02.553924 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b98c2-8a2b-4273-8f4d-9e52129dc686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:02.574334 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-1f7b98c2-8a2b-4273-8f4d-9e52129dc686 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:05:02.575541 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-1f7b98c2-8a2b-4273-8f4d-9e52129dc686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:05:02.627787 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:05:02.661908 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1057/2133] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:05:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:05:02.672424 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c09ef91-39d8-42ba-bc0f-4521c39ed2c8 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:02.674395 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-8c09ef91-39d8-42ba-bc0f-4521c39ed2c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:05:02.768030 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:05:02.773721 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c09ef91-39d8-42ba-bc0f-4521c39ed2c8 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:02.778412 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7efb85-66d4-4cef-984c-9636bdff4615 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:02.779217 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7efb85-66d4-4cef-984c-9636bdff4615 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:02.779987 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7efb85-66d4-4cef-984c-9636bdff4615 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:02.780255 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7efb85-66d4-4cef-984c-9636bdff4615 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:02.794081 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7efb85-66d4-4cef-984c-9636bdff4615 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:02.795661 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7efb85-66d4-4cef-984c-9636bdff4615 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:02.809813 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1077/2134] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:05:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:05:03.743720 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc1cce-a996-40dc-a976-e603cc3d43db None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:03.744229 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc1cce-a996-40dc-a976-e603cc3d43db None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:03.744307 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc1cce-a996-40dc-a976-e603cc3d43db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:03.783737 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782b134-5045-4fcf-9d8a-2ef423b3cf24 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:03.784718 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782b134-5045-4fcf-9d8a-2ef423b3cf24 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:03.785140 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782b134-5045-4fcf-9d8a-2ef423b3cf24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:03.803508 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-a8cc1cce-a996-40dc-a976-e603cc3d43db None None] MFA Rules not processed for user `de021dc74a4446d8b6a63883fd56a7c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:05:03.808061 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-a8cc1cce-a996-40dc-a976-e603cc3d43db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:05:03.818245 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-6782b134-5045-4fcf-9d8a-2ef423b3cf24 None None] MFA Rules not processed for user `7ce9ec4aea9449d3890f6dbb93f97968`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:05:03.820152 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-6782b134-5045-4fcf-9d8a-2ef423b3cf24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:05:03.942173 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1078/2135] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:05:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:05:03.959642 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6782b134-5045-4fcf-9d8a-2ef423b3cf24 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:03.961085 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-6782b134-5045-4fcf-9d8a-2ef423b3cf24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:05:03.973625 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1058/2136] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:05:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:05:03.997192 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cc1cce-a996-40dc-a976-e603cc3d43db None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:04.000778 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-a8cc1cce-a996-40dc-a976-e603cc3d43db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:05:04.064298 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6782b134-5045-4fcf-9d8a-2ef423b3cf24 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:04.068570 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de599037-ca51-4189-8975-430fe9096a99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:04.069236 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de599037-ca51-4189-8975-430fe9096a99 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:04.070002 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de599037-ca51-4189-8975-430fe9096a99 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:04.070002 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de599037-ca51-4189-8975-430fe9096a99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:04.084159 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de599037-ca51-4189-8975-430fe9096a99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:04.086198 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de599037-ca51-4189-8975-430fe9096a99 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:04.096316 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1079/2137] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:05:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:05:04.132315 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cc1cce-a996-40dc-a976-e603cc3d43db None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:04.139638 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52abb1c9-bf38-47d7-9bbe-24b5657bee13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:04.140511 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52abb1c9-bf38-47d7-9bbe-24b5657bee13 service nova] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:04.140903 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52abb1c9-bf38-47d7-9bbe-24b5657bee13 service nova] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:04.141287 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52abb1c9-bf38-47d7-9bbe-24b5657bee13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:04.151386 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52abb1c9-bf38-47d7-9bbe-24b5657bee13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:04.153152 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52abb1c9-bf38-47d7-9bbe-24b5657bee13 service nova] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:04.163296 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1059/2138] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:05:03 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:05:05.901237 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b6218-2f75-42f4-b548-40ce3598420b None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:05.901736 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b6218-2f75-42f4-b548-40ce3598420b None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:05.901841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b6218-2f75-42f4-b548-40ce3598420b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:05.932902 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-6c4b6218-2f75-42f4-b548-40ce3598420b None None] MFA Rules not processed for user `de021dc74a4446d8b6a63883fd56a7c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:05:05.934208 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-6c4b6218-2f75-42f4-b548-40ce3598420b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:05:05.951053 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1080/2139] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:05:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:05:06.847968 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6642b17c-f337-40ef-a272-1f987a158cc5 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:06.848438 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6642b17c-f337-40ef-a272-1f987a158cc5 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:06.848589 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6642b17c-f337-40ef-a272-1f987a158cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:06.893489 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-6642b17c-f337-40ef-a272-1f987a158cc5 None None] MFA Rules not processed for user `7ce9ec4aea9449d3890f6dbb93f97968`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:05:06.896226 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-6642b17c-f337-40ef-a272-1f987a158cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:05:06.914052 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1060/2140] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:05:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:05:12.974798 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98b9f7-69f1-493a-8e48-0f36f5eb48af None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:12.975241 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98b9f7-69f1-493a-8e48-0f36f5eb48af None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:12.975550 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98b9f7-69f1-493a-8e48-0f36f5eb48af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:12.998382 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-8e98b9f7-69f1-493a-8e48-0f36f5eb48af None None] MFA Rules not processed for user `86619a80df8841a3ba7d8471ac38056a`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:05:13.000382 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-8e98b9f7-69f1-493a-8e48-0f36f5eb48af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:05:13.082048 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1081/2141] 10.222.0.13 () {58 vars in 1009 bytes} [Tue Jun 7 22:05:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:05:13.151403 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6642b17c-f337-40ef-a272-1f987a158cc5 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:13.153915 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-6642b17c-f337-40ef-a272-1f987a158cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:05:13.233238 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:05:13.256036 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6642b17c-f337-40ef-a272-1f987a158cc5 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:13.263020 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d4bfd4-bd7b-45d7-8cb2-538a1cb2837a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1261563b6d4e50a4ffeea6a32bdb01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:13.263792 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4bfd4-bd7b-45d7-8cb2-538a1cb2837a service swift] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:13.264148 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4bfd4-bd7b-45d7-8cb2-538a1cb2837a service swift] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:13.264501 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4bfd4-bd7b-45d7-8cb2-538a1cb2837a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:13.273333 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d4bfd4-bd7b-45d7-8cb2-538a1cb2837a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:13.275513 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d4bfd4-bd7b-45d7-8cb2-538a1cb2837a service swift] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:13.284348 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1061/2142] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Jun 7 22:05:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 22:05:13.947276 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cbbcf2-a116-4bd9-8221-5ce12a1419e6 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:13.947603 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cbbcf2-a116-4bd9-8221-5ce12a1419e6 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:13.948037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cbbcf2-a116-4bd9-8221-5ce12a1419e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:13.966661 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-c8cbbcf2-a116-4bd9-8221-5ce12a1419e6 None None] MFA Rules not processed for user `86619a80df8841a3ba7d8471ac38056a`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:05:13.968009 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c8cbbcf2-a116-4bd9-8221-5ce12a1419e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:05:13.976534 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1082/2143] 10.222.0.13 () {58 vars in 967 bytes} [Tue Jun 7 22:05:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:05:13.989296 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d4bfd4-bd7b-45d7-8cb2-538a1cb2837a service swift] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:14.000476 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d4bfd4-bd7b-45d7-8cb2-538a1cb2837a service swift] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:14.005574 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa0838a-add6-4e15-808a-ca27a496b47f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1261563b6d4e50a4ffeea6a32bdb01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:14.006305 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0838a-add6-4e15-808a-ca27a496b47f service swift] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:14.006656 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0838a-add6-4e15-808a-ca27a496b47f service swift] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:14.007002 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0838a-add6-4e15-808a-ca27a496b47f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:14.016449 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa0838a-add6-4e15-808a-ca27a496b47f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:14.018099 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa0838a-add6-4e15-808a-ca27a496b47f service swift] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:14.026514 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1062/2144] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Jun 7 22:05:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 22:05:14.272391 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280858c-d525-45e0-9145-423ee6f54105 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:14.272684 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280858c-d525-45e0-9145-423ee6f54105 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:14.272982 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280858c-d525-45e0-9145-423ee6f54105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:14.290839 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-b280858c-d525-45e0-9145-423ee6f54105 None None] MFA Rules not processed for user `86619a80df8841a3ba7d8471ac38056a`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:05:14.292382 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-b280858c-d525-45e0-9145-423ee6f54105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:05:14.300397 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1083/2145] 10.222.0.13 () {58 vars in 1009 bytes} [Tue Jun 7 22:05:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:05:14.313229 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa0838a-add6-4e15-808a-ca27a496b47f service swift] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:14.323630 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa0838a-add6-4e15-808a-ca27a496b47f service swift] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:14.328597 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6b2401-e184-4a42-aef0-93d53747bed8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1261563b6d4e50a4ffeea6a32bdb01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:14.329230 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b2401-e184-4a42-aef0-93d53747bed8 service swift] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:14.329511 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b2401-e184-4a42-aef0-93d53747bed8 service swift] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:14.329790 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b2401-e184-4a42-aef0-93d53747bed8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:14.338509 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6b2401-e184-4a42-aef0-93d53747bed8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:14.340411 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6b2401-e184-4a42-aef0-93d53747bed8 service swift] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:14.349465 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1063/2146] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Jun 7 22:05:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 22:05:14.684403 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de9cf3-bf1d-42c5-a706-8ce5444b9593 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:14.684693 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de9cf3-bf1d-42c5-a706-8ce5444b9593 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:14.684976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de9cf3-bf1d-42c5-a706-8ce5444b9593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:14.703293 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-35de9cf3-bf1d-42c5-a706-8ce5444b9593 None None] MFA Rules not processed for user `86619a80df8841a3ba7d8471ac38056a`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:05:14.704623 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-35de9cf3-bf1d-42c5-a706-8ce5444b9593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:05:14.713494 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1084/2147] 10.222.0.13 () {58 vars in 967 bytes} [Tue Jun 7 22:05:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:05:14.725545 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6b2401-e184-4a42-aef0-93d53747bed8 service swift] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:14.737376 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6b2401-e184-4a42-aef0-93d53747bed8 service swift] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:14.747084 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e469085a-b2d7-4f54-a52d-409f17ca70a1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1261563b6d4e50a4ffeea6a32bdb01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:14.748457 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e469085a-b2d7-4f54-a52d-409f17ca70a1 service swift] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:14.748746 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e469085a-b2d7-4f54-a52d-409f17ca70a1 service swift] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:14.749026 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e469085a-b2d7-4f54-a52d-409f17ca70a1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:14.759250 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e469085a-b2d7-4f54-a52d-409f17ca70a1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:14.760949 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e469085a-b2d7-4f54-a52d-409f17ca70a1 service swift] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:14.770160 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1064/2148] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Jun 7 22:05:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 22:05:20.080646 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35de9cf3-bf1d-42c5-a706-8ce5444b9593 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:20.082129 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-35de9cf3-bf1d-42c5-a706-8ce5444b9593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:05:20.092081 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:05:20.097233 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35de9cf3-bf1d-42c5-a706-8ce5444b9593 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:20.102538 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5de9aa9-2c61-49c0-9a8f-4866563b9eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:20.103420 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de9aa9-2c61-49c0-9a8f-4866563b9eda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:20.103701 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de9aa9-2c61-49c0-9a8f-4866563b9eda admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:20.103971 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de9aa9-2c61-49c0-9a8f-4866563b9eda admin admin] PATH_INFO: `/v3/users/d03864707f284a14ad9726c4023f32b8` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:20.105170 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5de9aa9-2c61-49c0-9a8f-4866563b9eda admin admin] RBAC: Authorizing `identity:delete_user(user_id=d03864707f284a14ad9726c4023f32b8)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:20.107083 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5de9aa9-2c61-49c0-9a8f-4866563b9eda admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:20.155705 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-e5de9aa9-2c61-49c0-9a8f-4866563b9eda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd03864707f284a14ad9726c4023f32b8'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:05:20.163013 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-e5de9aa9-2c61-49c0-9a8f-4866563b9eda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd03864707f284a14ad9726c4023f32b8'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:05:20.175964 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-e5de9aa9-2c61-49c0-9a8f-4866563b9eda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd03864707f284a14ad9726c4023f32b8'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:05:20.176680 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-e5de9aa9-2c61-49c0-9a8f-4866563b9eda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd03864707f284a14ad9726c4023f32b8'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:05:20.177278 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-e5de9aa9-2c61-49c0-9a8f-4866563b9eda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd03864707f284a14ad9726c4023f32b8'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:05:20.190551 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1085/2149] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:05:20 2022] DELETE /identity/v3/users/d03864707f284a14ad9726c4023f32b8 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:05:20.195055 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe834798-2e04-4e41-ada7-df99bfafbb17 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:20.195577 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe834798-2e04-4e41-ada7-df99bfafbb17 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:20.195931 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe834798-2e04-4e41-ada7-df99bfafbb17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:20.216209 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-fe834798-2e04-4e41-ada7-df99bfafbb17 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:05:20.217432 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-fe834798-2e04-4e41-ada7-df99bfafbb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:05:20.276904 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1065/2150] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:05:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:05:20.286422 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5de9aa9-2c61-49c0-9a8f-4866563b9eda admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:20.287851 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-e5de9aa9-2c61-49c0-9a8f-4866563b9eda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:05:20.343605 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5de9aa9-2c61-49c0-9a8f-4866563b9eda admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:20.347253 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a88d45-c875-457d-a448-a328558753f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:20.347920 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a88d45-c875-457d-a448-a328558753f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:20.348203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a88d45-c875-457d-a448-a328558753f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:20.348479 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a88d45-c875-457d-a448-a328558753f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:20.358449 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a88d45-c875-457d-a448-a328558753f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:20.359904 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a88d45-c875-457d-a448-a328558753f0 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:20.369383 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1086/2151] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:05:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:05:20.541769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe834798-2e04-4e41-ada7-df99bfafbb17 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:20.543256 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-fe834798-2e04-4e41-ada7-df99bfafbb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:05:20.557689 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe834798-2e04-4e41-ada7-df99bfafbb17 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:20.562637 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6549fad-9e16-4c35-b57a-569ed59e31f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:20.563546 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6549fad-9e16-4c35-b57a-569ed59e31f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:20.563980 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6549fad-9e16-4c35-b57a-569ed59e31f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:20.564407 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6549fad-9e16-4c35-b57a-569ed59e31f5 admin admin] PATH_INFO: `/v3/projects/f240f9b198124b58826697af9f8ab205` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:20.565735 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6549fad-9e16-4c35-b57a-569ed59e31f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f240f9b198124b58826697af9f8ab205)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:20.567805 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6549fad-9e16-4c35-b57a-569ed59e31f5 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:20.624566 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-b6549fad-9e16-4c35-b57a-569ed59e31f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f240f9b198124b58826697af9f8ab205', 'request_id': 'req-b6549fad-9e16-4c35-b57a-569ed59e31f5'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:05:20.633564 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-b6549fad-9e16-4c35-b57a-569ed59e31f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f240f9b198124b58826697af9f8ab205', 'request_id': 'req-b6549fad-9e16-4c35-b57a-569ed59e31f5'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:05:20.641691 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-b6549fad-9e16-4c35-b57a-569ed59e31f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f240f9b198124b58826697af9f8ab205', 'request_id': 'req-b6549fad-9e16-4c35-b57a-569ed59e31f5'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:05:20.655514 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-b6549fad-9e16-4c35-b57a-569ed59e31f5 admin admin] The token cache is being invalidate because project f240f9b198124b58826697af9f8ab205 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:05:20.655801 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-b6549fad-9e16-4c35-b57a-569ed59e31f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:05:20.656486 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-b6549fad-9e16-4c35-b57a-569ed59e31f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:05:20.658422 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1066/2152] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:05:20 2022] DELETE /identity/v3/projects/f240f9b198124b58826697af9f8ab205 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:05:20.663388 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8512f4eb-09e4-469f-9485-83b761b70596 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:20.663835 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8512f4eb-09e4-469f-9485-83b761b70596 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:20.664179 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8512f4eb-09e4-469f-9485-83b761b70596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:20.682827 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-8512f4eb-09e4-469f-9485-83b761b70596 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:05:20.684081 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-8512f4eb-09e4-469f-9485-83b761b70596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:05:20.748480 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1087/2153] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:05:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:05:20.762030 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3326abd6-837e-4806-a5e0-cba13b5040c4 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:20.762030 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3326abd6-837e-4806-a5e0-cba13b5040c4 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:20.762030 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3326abd6-837e-4806-a5e0-cba13b5040c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:20.781114 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-3326abd6-837e-4806-a5e0-cba13b5040c4 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:05:20.782420 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-3326abd6-837e-4806-a5e0-cba13b5040c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:05:20.790333 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1067/2154] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:05:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:05:20.794633 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8512f4eb-09e4-469f-9485-83b761b70596 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:20.805310 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8512f4eb-09e4-469f-9485-83b761b70596 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:20.810801 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67652c0f-0ac2-4223-9081-77d280ba4283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:20.811569 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67652c0f-0ac2-4223-9081-77d280ba4283 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:20.811926 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67652c0f-0ac2-4223-9081-77d280ba4283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:20.812211 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67652c0f-0ac2-4223-9081-77d280ba4283 admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:20.813283 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67652c0f-0ac2-4223-9081-77d280ba4283 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:05:20.813283 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67652c0f-0ac2-4223-9081-77d280ba4283 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:20.814874 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67652c0f-0ac2-4223-9081-77d280ba4283 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:20.829580 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1088/2155] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:05:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:05:20.834100 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3326abd6-837e-4806-a5e0-cba13b5040c4 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:20.848455 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3326abd6-837e-4806-a5e0-cba13b5040c4 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:20.853692 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861ebbb6-9e3e-4029-96c4-4fee29a3c509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:20.854786 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ebbb6-9e3e-4029-96c4-4fee29a3c509 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:20.854786 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ebbb6-9e3e-4029-96c4-4fee29a3c509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:20.855156 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ebbb6-9e3e-4029-96c4-4fee29a3c509 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:20.855971 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861ebbb6-9e3e-4029-96c4-4fee29a3c509 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:20.857968 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861ebbb6-9e3e-4029-96c4-4fee29a3c509 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:20.875189 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:05:20.893525 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1068/2156] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:05:20 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:05:20.897578 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67652c0f-0ac2-4223-9081-77d280ba4283 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:20.904794 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:05:20.909598 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67652c0f-0ac2-4223-9081-77d280ba4283 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:20.971069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e32af5-004d-4b67-bdd7-4f48a1678d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:20.971679 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e32af5-004d-4b67-bdd7-4f48a1678d8e admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:20.971966 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e32af5-004d-4b67-bdd7-4f48a1678d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:20.972245 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e32af5-004d-4b67-bdd7-4f48a1678d8e admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:20.972949 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e32af5-004d-4b67-bdd7-4f48a1678d8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:20.974880 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e32af5-004d-4b67-bdd7-4f48a1678d8e admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:20.996908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1089/2157] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:05:20 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:05:21.001112 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861ebbb6-9e3e-4029-96c4-4fee29a3c509 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:21.012439 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861ebbb6-9e3e-4029-96c4-4fee29a3c509 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:21.017183 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e953a958-a3ff-4998-94b2-6e3c68b0cc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:21.017861 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e953a958-a3ff-4998-94b2-6e3c68b0cc92 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:21.018140 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e953a958-a3ff-4998-94b2-6e3c68b0cc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:21.018415 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e953a958-a3ff-4998-94b2-6e3c68b0cc92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:21.028244 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e953a958-a3ff-4998-94b2-6e3c68b0cc92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:21.031978 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e953a958-a3ff-4998-94b2-6e3c68b0cc92 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:21.067354 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1069/2158] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:05:21 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:05:21.072571 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e32af5-004d-4b67-bdd7-4f48a1678d8e admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:21.086512 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e32af5-004d-4b67-bdd7-4f48a1678d8e admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:21.093121 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b1a332-53a1-467c-b175-3704323cf3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:21.093980 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b1a332-53a1-467c-b175-3704323cf3f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:21.095233 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b1a332-53a1-467c-b175-3704323cf3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:21.097673 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b1a332-53a1-467c-b175-3704323cf3f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:21.099131 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b1a332-53a1-467c-b175-3704323cf3f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:21.101664 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b1a332-53a1-467c-b175-3704323cf3f2 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:21.114997 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1090/2159] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:05:21 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:05:21.120474 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e953a958-a3ff-4998-94b2-6e3c68b0cc92 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:21.139214 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e953a958-a3ff-4998-94b2-6e3c68b0cc92 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:21.152172 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971077ea-5ee9-473d-a594-d5eca1a91df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:21.153096 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971077ea-5ee9-473d-a594-d5eca1a91df1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:21.153381 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971077ea-5ee9-473d-a594-d5eca1a91df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:21.153682 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971077ea-5ee9-473d-a594-d5eca1a91df1 admin admin] PATH_INFO: `/v3/projects/bd31cd38d46643fbac516d21ee4f89ac/users/7af02cd6ffd0484c8f6569629cd6313c/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:21.180517 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971077ea-5ee9-473d-a594-d5eca1a91df1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd31cd38d46643fbac516d21ee4f89ac, user_id=7af02cd6ffd0484c8f6569629cd6313c, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:21.185114 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971077ea-5ee9-473d-a594-d5eca1a91df1 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:21.226232 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb7370e-0746-4561-8e01-c9bfc932ab69 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:21.226427 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1070/2160] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:05:21 2022] PUT /identity/v3/projects/bd31cd38d46643fbac516d21ee4f89ac/users/7af02cd6ffd0484c8f6569629cd6313c/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:05:21.228110 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb7370e-0746-4561-8e01-c9bfc932ab69 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:21.228516 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb7370e-0746-4561-8e01-c9bfc932ab69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:21.259677 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-efb7370e-0746-4561-8e01-c9bfc932ab69 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:05:21.261246 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-efb7370e-0746-4561-8e01-c9bfc932ab69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:05:21.339664 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1091/2161] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:05:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:05:21.377545 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971077ea-5ee9-473d-a594-d5eca1a91df1 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:21.380373 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-971077ea-5ee9-473d-a594-d5eca1a91df1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:05:21.462324 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971077ea-5ee9-473d-a594-d5eca1a91df1 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:21.466586 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b1aeed-c4b0-430d-a69b-62968d5ebee9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:21.467536 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b1aeed-c4b0-430d-a69b-62968d5ebee9 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:21.467890 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b1aeed-c4b0-430d-a69b-62968d5ebee9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:21.468243 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b1aeed-c4b0-430d-a69b-62968d5ebee9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:21.478736 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b1aeed-c4b0-430d-a69b-62968d5ebee9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:21.480162 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b1aeed-c4b0-430d-a69b-62968d5ebee9 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:21.490248 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1071/2162] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:05:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:05:26.522441 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cff1a4-f549-4536-9710-9d6f467f4c29 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:26.522918 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cff1a4-f549-4536-9710-9d6f467f4c29 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:26.523318 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cff1a4-f549-4536-9710-9d6f467f4c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:26.594145 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-04cff1a4-f549-4536-9710-9d6f467f4c29 None None] MFA Rules not processed for user `7af02cd6ffd0484c8f6569629cd6313c`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:05:26.596664 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-04cff1a4-f549-4536-9710-9d6f467f4c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:05:26.694507 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:05:26.715562 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1092/2163] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:05:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:05:26.720015 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b1aeed-c4b0-430d-a69b-62968d5ebee9 service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:26.771601 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b1aeed-c4b0-430d-a69b-62968d5ebee9 service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:26.776986 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e769da-564e-4f8f-8cde-137877e4e53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:26.777640 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e769da-564e-4f8f-8cde-137877e4e53a admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:26.777927 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e769da-564e-4f8f-8cde-137877e4e53a admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:26.778206 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e769da-564e-4f8f-8cde-137877e4e53a admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:26.778894 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e769da-564e-4f8f-8cde-137877e4e53a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:26.780843 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e769da-564e-4f8f-8cde-137877e4e53a admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:26.818439 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1072/2164] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:05:26 2022] POST /identity/v3/projects => generated 383 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:05:26.823400 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cff1a4-f549-4536-9710-9d6f467f4c29 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:26.837263 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cff1a4-f549-4536-9710-9d6f467f4c29 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:26.895102 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b856e6dd-dae8-4190-a313-f46999f8505f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:26.895810 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b856e6dd-dae8-4190-a313-f46999f8505f admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:26.896098 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b856e6dd-dae8-4190-a313-f46999f8505f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:26.896378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b856e6dd-dae8-4190-a313-f46999f8505f admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:26.897081 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b856e6dd-dae8-4190-a313-f46999f8505f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:26.899059 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b856e6dd-dae8-4190-a313-f46999f8505f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:26.918482 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1093/2165] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:05:26 2022] POST /identity/v3/users => generated 298 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:05:26.922506 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e769da-564e-4f8f-8cde-137877e4e53a admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:26.933348 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e769da-564e-4f8f-8cde-137877e4e53a admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:26.938008 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f286842-2969-4902-932a-405235baa350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:26.938670 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f286842-2969-4902-932a-405235baa350 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:26.938955 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f286842-2969-4902-932a-405235baa350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:26.939235 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f286842-2969-4902-932a-405235baa350 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:26.940005 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f286842-2969-4902-932a-405235baa350 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:26.941589 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f286842-2969-4902-932a-405235baa350 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:26.952472 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1073/2166] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:05:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:05:26.956215 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b856e6dd-dae8-4190-a313-f46999f8505f admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:26.966223 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b856e6dd-dae8-4190-a313-f46999f8505f admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:26.970866 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b2b65f-6f51-44b4-b48e-9a8c1d2964e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:26.971771 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b2b65f-6f51-44b4-b48e-9a8c1d2964e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:26.972055 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b2b65f-6f51-44b4-b48e-9a8c1d2964e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:26.972335 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b2b65f-6f51-44b4-b48e-9a8c1d2964e9 admin admin] PATH_INFO: `/v3/projects/7ba15b864eac4c0081b6edc6f8f45105/users/0388de1feaf74c9b8073f04b276dc310/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:26.986987 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b2b65f-6f51-44b4-b48e-9a8c1d2964e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ba15b864eac4c0081b6edc6f8f45105, user_id=0388de1feaf74c9b8073f04b276dc310, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:26.989675 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b2b65f-6f51-44b4-b48e-9a8c1d2964e9 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:27.003540 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1094/2167] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:05:26 2022] PUT /identity/v3/projects/7ba15b864eac4c0081b6edc6f8f45105/users/0388de1feaf74c9b8073f04b276dc310/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:05:27.007412 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f286842-2969-4902-932a-405235baa350 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:27.018052 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f286842-2969-4902-932a-405235baa350 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:27.051899 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:05:27.079030 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8900fdc-5734-47b5-b26f-ae03c4ec8d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:27.079773 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8900fdc-5734-47b5-b26f-ae03c4ec8d04 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:27.080057 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8900fdc-5734-47b5-b26f-ae03c4ec8d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:27.080333 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8900fdc-5734-47b5-b26f-ae03c4ec8d04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:27.081120 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8900fdc-5734-47b5-b26f-ae03c4ec8d04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:27.082893 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8900fdc-5734-47b5-b26f-ae03c4ec8d04 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:27.093487 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1074/2168] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:05:27 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:05:27.097752 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b2b65f-6f51-44b4-b48e-9a8c1d2964e9 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:27.108570 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b2b65f-6f51-44b4-b48e-9a8c1d2964e9 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:27.113280 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5c82b1-2f59-4ee2-85cf-728a5d37d930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:27.114153 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c82b1-2f59-4ee2-85cf-728a5d37d930 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:27.114435 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c82b1-2f59-4ee2-85cf-728a5d37d930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:27.114711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c82b1-2f59-4ee2-85cf-728a5d37d930 admin admin] PATH_INFO: `/v3/projects/7ba15b864eac4c0081b6edc6f8f45105/users/0388de1feaf74c9b8073f04b276dc310/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:27.116695 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5c82b1-2f59-4ee2-85cf-728a5d37d930 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ba15b864eac4c0081b6edc6f8f45105, user_id=0388de1feaf74c9b8073f04b276dc310, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:27.120221 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5c82b1-2f59-4ee2-85cf-728a5d37d930 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:27.131111 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1095/2169] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:05:27 2022] PUT /identity/v3/projects/7ba15b864eac4c0081b6edc6f8f45105/users/0388de1feaf74c9b8073f04b276dc310/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:05:32.693165 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9637fe-81d0-4437-ab21-175d07e9a8e9 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:32.693695 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9637fe-81d0-4437-ab21-175d07e9a8e9 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:32.693695 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9637fe-81d0-4437-ab21-175d07e9a8e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:32.734555 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-3f9637fe-81d0-4437-ab21-175d07e9a8e9 None None] MFA Rules not processed for user `0388de1feaf74c9b8073f04b276dc310`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:05:32.736024 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-3f9637fe-81d0-4437-ab21-175d07e9a8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:05:32.830594 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1075/2170] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:05:32 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:05:32.842372 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5c82b1-2f59-4ee2-85cf-728a5d37d930 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:32.844071 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-3f5c82b1-2f59-4ee2-85cf-728a5d37d930 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:05:32.915950 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5c82b1-2f59-4ee2-85cf-728a5d37d930 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:32.919491 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b7f393-6241-492b-b07f-b9728aa8afe0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96987b04471944bc86ab44ff9936c732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:32.920131 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b7f393-6241-492b-b07f-b9728aa8afe0 service glance] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:32.920413 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b7f393-6241-492b-b07f-b9728aa8afe0 service glance] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:32.920687 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b7f393-6241-492b-b07f-b9728aa8afe0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:32.958500 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:05:32.979108 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b7f393-6241-492b-b07f-b9728aa8afe0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:32.980692 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b7f393-6241-492b-b07f-b9728aa8afe0 service glance] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:32.990432 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1096/2171] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:05:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:05:33.090346 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e659d-6fa5-453f-8e9a-e1008c42d3ca None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:33.090638 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e659d-6fa5-453f-8e9a-e1008c42d3ca None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:33.090920 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e659d-6fa5-453f-8e9a-e1008c42d3ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:33.114022 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-e81e659d-6fa5-453f-8e9a-e1008c42d3ca None None] MFA Rules not processed for user `86619a80df8841a3ba7d8471ac38056a`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:05:33.115288 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-e81e659d-6fa5-453f-8e9a-e1008c42d3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:05:33.177317 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1076/2172] 10.222.0.13 () {58 vars in 1009 bytes} [Tue Jun 7 22:05:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:05:33.191257 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b7f393-6241-492b-b07f-b9728aa8afe0 service glance] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:33.193395 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-05b7f393-6241-492b-b07f-b9728aa8afe0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:05:33.369942 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b7f393-6241-492b-b07f-b9728aa8afe0 service glance] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:33.375921 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964596e6-506c-4678-b8bb-db357124b7fb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1261563b6d4e50a4ffeea6a32bdb01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:33.376710 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964596e6-506c-4678-b8bb-db357124b7fb service swift] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:33.377305 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964596e6-506c-4678-b8bb-db357124b7fb service swift] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:33.377468 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964596e6-506c-4678-b8bb-db357124b7fb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:33.408946 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964596e6-506c-4678-b8bb-db357124b7fb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:33.411761 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964596e6-506c-4678-b8bb-db357124b7fb service swift] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:33.434614 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1097/2173] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Jun 7 22:05:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 22:05:37.051549 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81e659d-6fa5-453f-8e9a-e1008c42d3ca None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:37.053950 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-e81e659d-6fa5-453f-8e9a-e1008c42d3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:05:37.218928 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81e659d-6fa5-453f-8e9a-e1008c42d3ca None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:37.223812 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08005a7c-9a62-47a7-81f8-c245d2e915b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:37.224780 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08005a7c-9a62-47a7-81f8-c245d2e915b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:37.225145 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08005a7c-9a62-47a7-81f8-c245d2e915b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:37.225503 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08005a7c-9a62-47a7-81f8-c245d2e915b5 admin admin] PATH_INFO: `/v3/users/e9955cc2c9fe47a3b0333133706eec7d` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:37.227865 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08005a7c-9a62-47a7-81f8-c245d2e915b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9955cc2c9fe47a3b0333133706eec7d)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:37.229952 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08005a7c-9a62-47a7-81f8-c245d2e915b5 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:37.237935 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:05:37.310979 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-08005a7c-9a62-47a7-81f8-c245d2e915b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9955cc2c9fe47a3b0333133706eec7d'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:05:37.311813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-08005a7c-9a62-47a7-81f8-c245d2e915b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9955cc2c9fe47a3b0333133706eec7d'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:05:37.332043 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-08005a7c-9a62-47a7-81f8-c245d2e915b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9955cc2c9fe47a3b0333133706eec7d'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:05:37.357105 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-08005a7c-9a62-47a7-81f8-c245d2e915b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9955cc2c9fe47a3b0333133706eec7d'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:05:37.372768 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-08005a7c-9a62-47a7-81f8-c245d2e915b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9955cc2c9fe47a3b0333133706eec7d'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:05:37.388778 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1077/2174] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:05:37 2022] DELETE /identity/v3/users/e9955cc2c9fe47a3b0333133706eec7d => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:05:37.390927 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e0795-ee3b-47ad-9874-44c7ead1ecc7 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:37.391467 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e0795-ee3b-47ad-9874-44c7ead1ecc7 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:37.391563 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e0795-ee3b-47ad-9874-44c7ead1ecc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:37.428307 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-f66e0795-ee3b-47ad-9874-44c7ead1ecc7 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:05:37.430742 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-f66e0795-ee3b-47ad-9874-44c7ead1ecc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:05:37.526169 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1098/2175] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:05:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:05:37.527228 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08005a7c-9a62-47a7-81f8-c245d2e915b5 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:37.528772 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-08005a7c-9a62-47a7-81f8-c245d2e915b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:05:37.677436 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08005a7c-9a62-47a7-81f8-c245d2e915b5 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:37.681936 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545fced1-d72b-4ed6-ad08-abb9544efe7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:37.683399 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545fced1-d72b-4ed6-ad08-abb9544efe7a service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:37.683399 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545fced1-d72b-4ed6-ad08-abb9544efe7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:37.683399 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545fced1-d72b-4ed6-ad08-abb9544efe7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:37.697320 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545fced1-d72b-4ed6-ad08-abb9544efe7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:37.698823 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545fced1-d72b-4ed6-ad08-abb9544efe7a service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:37.714844 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1078/2176] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:05:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 22:05:38.295203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e0795-ee3b-47ad-9874-44c7ead1ecc7 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:38.303203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-f66e0795-ee3b-47ad-9874-44c7ead1ecc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:05:38.330696 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e0795-ee3b-47ad-9874-44c7ead1ecc7 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:38.352963 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f157b-1bd5-44eb-957d-f7e2ef9d6cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:38.353870 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f157b-1bd5-44eb-957d-f7e2ef9d6cf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:38.365279 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f157b-1bd5-44eb-957d-f7e2ef9d6cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:38.365696 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f157b-1bd5-44eb-957d-f7e2ef9d6cf1 admin admin] PATH_INFO: `/v3/projects/5d15f11ecfdf49c789f948394783b61e` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:38.377676 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459f157b-1bd5-44eb-957d-f7e2ef9d6cf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d15f11ecfdf49c789f948394783b61e)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:38.379899 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459f157b-1bd5-44eb-957d-f7e2ef9d6cf1 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:38.473931 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-459f157b-1bd5-44eb-957d-f7e2ef9d6cf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d15f11ecfdf49c789f948394783b61e', 'request_id': 'req-459f157b-1bd5-44eb-957d-f7e2ef9d6cf1'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:05:38.514817 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-459f157b-1bd5-44eb-957d-f7e2ef9d6cf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d15f11ecfdf49c789f948394783b61e', 'request_id': 'req-459f157b-1bd5-44eb-957d-f7e2ef9d6cf1'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:05:38.520144 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:05:38.533757 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-459f157b-1bd5-44eb-957d-f7e2ef9d6cf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d15f11ecfdf49c789f948394783b61e', 'request_id': 'req-459f157b-1bd5-44eb-957d-f7e2ef9d6cf1'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:05:38.556339 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-459f157b-1bd5-44eb-957d-f7e2ef9d6cf1 admin admin] The token cache is being invalidate because project 5d15f11ecfdf49c789f948394783b61e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:05:38.557156 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-459f157b-1bd5-44eb-957d-f7e2ef9d6cf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:05:38.558069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-459f157b-1bd5-44eb-957d-f7e2ef9d6cf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:05:38.560264 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1099/2177] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:05:38 2022] DELETE /identity/v3/projects/5d15f11ecfdf49c789f948394783b61e => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:05:38.567010 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63865cec-6f25-4256-a07a-3b4b3a0a4051 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:38.567659 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63865cec-6f25-4256-a07a-3b4b3a0a4051 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:38.567659 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63865cec-6f25-4256-a07a-3b4b3a0a4051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:38.644360 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-63865cec-6f25-4256-a07a-3b4b3a0a4051 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:05:38.645617 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-63865cec-6f25-4256-a07a-3b4b3a0a4051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:05:38.708018 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:05:38.796730 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1079/2178] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:05:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:05:38.799083 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7a8be-5c72-47f1-91cb-7716303ae299 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:38.810295 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7a8be-5c72-47f1-91cb-7716303ae299 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:38.810295 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7a8be-5c72-47f1-91cb-7716303ae299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:38.835724 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-43e7a8be-5c72-47f1-91cb-7716303ae299 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:05:38.835724 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-43e7a8be-5c72-47f1-91cb-7716303ae299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:05:38.855712 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1100/2179] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:05:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:05:38.859817 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63865cec-6f25-4256-a07a-3b4b3a0a4051 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:38.874175 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63865cec-6f25-4256-a07a-3b4b3a0a4051 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:38.891464 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ea8098-681d-4298-966b-e9dda1745288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:38.892373 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea8098-681d-4298-966b-e9dda1745288 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:38.892811 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea8098-681d-4298-966b-e9dda1745288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:38.892811 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea8098-681d-4298-966b-e9dda1745288 admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:38.893430 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ea8098-681d-4298-966b-e9dda1745288 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:05:38.893758 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ea8098-681d-4298-966b-e9dda1745288 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:38.899766 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ea8098-681d-4298-966b-e9dda1745288 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:38.919617 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1080/2180] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:05:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 22:05:38.924050 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e7a8be-5c72-47f1-91cb-7716303ae299 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:38.937518 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e7a8be-5c72-47f1-91cb-7716303ae299 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:38.943353 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b2e3a7-73f0-4164-8d18-115f5ab08f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:38.945287 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2e3a7-73f0-4164-8d18-115f5ab08f27 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:38.945708 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2e3a7-73f0-4164-8d18-115f5ab08f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:38.946146 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2e3a7-73f0-4164-8d18-115f5ab08f27 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:38.946994 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b2e3a7-73f0-4164-8d18-115f5ab08f27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:38.949344 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b2e3a7-73f0-4164-8d18-115f5ab08f27 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:38.984554 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1101/2181] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:05:38 2022] POST /identity/v3/projects => generated 377 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:05:38.988324 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ea8098-681d-4298-966b-e9dda1745288 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:39.012148 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ea8098-681d-4298-966b-e9dda1745288 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:39.087664 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b44e3-d3e9-4b8e-b395-72f18ab32cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:39.088435 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b44e3-d3e9-4b8e-b395-72f18ab32cb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:39.088817 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b44e3-d3e9-4b8e-b395-72f18ab32cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:39.089188 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b44e3-d3e9-4b8e-b395-72f18ab32cb4 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:39.090038 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1b44e3-d3e9-4b8e-b395-72f18ab32cb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:39.092112 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1b44e3-d3e9-4b8e-b395-72f18ab32cb4 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:39.132822 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1081/2182] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:05:38 2022] POST /identity/v3/users => generated 289 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:05:39.134390 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b2e3a7-73f0-4164-8d18-115f5ab08f27 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:39.149983 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b2e3a7-73f0-4164-8d18-115f5ab08f27 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:39.155509 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1941b9ae-37b8-468b-b74d-7e61368348a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:39.156193 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1941b9ae-37b8-468b-b74d-7e61368348a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:39.156479 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1941b9ae-37b8-468b-b74d-7e61368348a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:39.156757 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1941b9ae-37b8-468b-b74d-7e61368348a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:39.157500 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1941b9ae-37b8-468b-b74d-7e61368348a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:39.159127 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1941b9ae-37b8-468b-b74d-7e61368348a2 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:39.174917 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1102/2183] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:05:39.176197 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b44e3-d3e9-4b8e-b395-72f18ab32cb4 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:39.190249 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b44e3-d3e9-4b8e-b395-72f18ab32cb4 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:39.196922 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4399ca-7510-4bc5-9a3e-c38fd542c2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:39.197593 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4399ca-7510-4bc5-9a3e-c38fd542c2dd admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:39.198020 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4399ca-7510-4bc5-9a3e-c38fd542c2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:39.198305 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4399ca-7510-4bc5-9a3e-c38fd542c2dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:39.199055 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4399ca-7510-4bc5-9a3e-c38fd542c2dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:39.200668 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4399ca-7510-4bc5-9a3e-c38fd542c2dd admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:39.212992 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1082/2184] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:05:39.218607 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1941b9ae-37b8-468b-b74d-7e61368348a2 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:39.230398 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1941b9ae-37b8-468b-b74d-7e61368348a2 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:39.238146 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b61e70-3964-45a9-b31d-a21885a5c476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:39.239069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b61e70-3964-45a9-b31d-a21885a5c476 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:39.239378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b61e70-3964-45a9-b31d-a21885a5c476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:39.239688 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b61e70-3964-45a9-b31d-a21885a5c476 admin admin] PATH_INFO: `/v3/projects/579eb78d8af74e2cadce3a0c793b4440/users/c710942802df49ec8bd630e4b225eed2/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:39.259468 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b61e70-3964-45a9-b31d-a21885a5c476 admin admin] RBAC: Authorizing `identity:create_grant(project_id=579eb78d8af74e2cadce3a0c793b4440, user_id=c710942802df49ec8bd630e4b225eed2, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:39.262341 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b61e70-3964-45a9-b31d-a21885a5c476 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:39.283381 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1103/2185] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:05:39 2022] PUT /identity/v3/projects/579eb78d8af74e2cadce3a0c793b4440/users/c710942802df49ec8bd630e4b225eed2/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:05:39.285739 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d0dfc-28c9-4289-af1a-06616a45b142 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:39.286106 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d0dfc-28c9-4289-af1a-06616a45b142 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:39.286479 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d0dfc-28c9-4289-af1a-06616a45b142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:39.308013 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-e47d0dfc-28c9-4289-af1a-06616a45b142 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:05:39.309323 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-e47d0dfc-28c9-4289-af1a-06616a45b142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:05:39.411304 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1083/2186] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:05:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:05:39.422531 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b61e70-3964-45a9-b31d-a21885a5c476 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:39.425206 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-f7b61e70-3964-45a9-b31d-a21885a5c476 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:05:39.511812 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b61e70-3964-45a9-b31d-a21885a5c476 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:39.518236 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e6d0e-8ab5-4996-85aa-149ec9f50fdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:39.520084 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e6d0e-8ab5-4996-85aa-149ec9f50fdf service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:39.520369 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e6d0e-8ab5-4996-85aa-149ec9f50fdf service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:39.520642 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e6d0e-8ab5-4996-85aa-149ec9f50fdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:39.532162 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738e6d0e-8ab5-4996-85aa-149ec9f50fdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:39.533596 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738e6d0e-8ab5-4996-85aa-149ec9f50fdf service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:39.545470 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1104/2187] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:05:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:05:44.856471 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ca2a6-16f2-40fc-b371-868d2500b8b5 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:44.857019 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ca2a6-16f2-40fc-b371-868d2500b8b5 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:44.857109 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ca2a6-16f2-40fc-b371-868d2500b8b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:44.891596 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-c88ca2a6-16f2-40fc-b371-868d2500b8b5 None None] MFA Rules not processed for user `7ce9ec4aea9449d3890f6dbb93f97968`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:05:44.892925 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-c88ca2a6-16f2-40fc-b371-868d2500b8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:05:44.968235 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:05:44.988485 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1084/2188] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:05:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:05:47.230023 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bce8ab-6c89-4e4c-bee4-763e9fed726f None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:47.231077 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bce8ab-6c89-4e4c-bee4-763e9fed726f None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:47.231453 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bce8ab-6c89-4e4c-bee4-763e9fed726f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:47.261917 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-c3bce8ab-6c89-4e4c-bee4-763e9fed726f None None] MFA Rules not processed for user `c710942802df49ec8bd630e4b225eed2`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:05:47.263489 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c3bce8ab-6c89-4e4c-bee4-763e9fed726f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:05:47.388681 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1105/2189] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:05:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:05:47.408759 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88ca2a6-16f2-40fc-b371-868d2500b8b5 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:47.442337 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88ca2a6-16f2-40fc-b371-868d2500b8b5 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:47.451508 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f849ea5-e56b-4828-9219-ab022f2821e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:47.452408 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f849ea5-e56b-4828-9219-ab022f2821e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:47.452861 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f849ea5-e56b-4828-9219-ab022f2821e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:47.453291 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f849ea5-e56b-4828-9219-ab022f2821e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:47.458049 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f849ea5-e56b-4828-9219-ab022f2821e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:47.463026 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f849ea5-e56b-4828-9219-ab022f2821e6 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:47.539587 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bce8ab-6c89-4e4c-bee4-763e9fed726f None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:47.539817 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1085/2190] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:05:47 2022] POST /identity/v3/projects => generated 379 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:05:47.558124 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bce8ab-6c89-4e4c-bee4-763e9fed726f None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:47.570358 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:05:47.662672 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c1db4-a102-404e-8c94-a04d0af9e9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:47.663502 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c1db4-a102-404e-8c94-a04d0af9e9db admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:47.663789 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c1db4-a102-404e-8c94-a04d0af9e9db admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:47.664068 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c1db4-a102-404e-8c94-a04d0af9e9db admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:47.664759 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9c1db4-a102-404e-8c94-a04d0af9e9db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:47.666677 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9c1db4-a102-404e-8c94-a04d0af9e9db admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:47.688385 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1106/2191] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:05:47 2022] POST /identity/v3/users => generated 296 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:05:47.692108 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f849ea5-e56b-4828-9219-ab022f2821e6 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:47.704740 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f849ea5-e56b-4828-9219-ab022f2821e6 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:47.710076 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b239008-3c5d-4996-a7aa-732d7e0d8f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:47.711092 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b239008-3c5d-4996-a7aa-732d7e0d8f49 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:47.711305 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b239008-3c5d-4996-a7aa-732d7e0d8f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:47.711612 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b239008-3c5d-4996-a7aa-732d7e0d8f49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:47.712340 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b239008-3c5d-4996-a7aa-732d7e0d8f49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:47.713993 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b239008-3c5d-4996-a7aa-732d7e0d8f49 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:47.726563 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1086/2192] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:05:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:05:47.730135 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c1db4-a102-404e-8c94-a04d0af9e9db admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:47.743157 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c1db4-a102-404e-8c94-a04d0af9e9db admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:47.748637 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491cad0e-16a2-456a-aa8d-7d1238acbf2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:47.749732 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491cad0e-16a2-456a-aa8d-7d1238acbf2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:47.750016 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491cad0e-16a2-456a-aa8d-7d1238acbf2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:47.750306 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491cad0e-16a2-456a-aa8d-7d1238acbf2e admin admin] PATH_INFO: `/v3/projects/0a1a5d620d604e468fcf86e5cdd09467/users/61973960102c4341889b29825fb38940/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:47.773986 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491cad0e-16a2-456a-aa8d-7d1238acbf2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a1a5d620d604e468fcf86e5cdd09467, user_id=61973960102c4341889b29825fb38940, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:47.777032 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491cad0e-16a2-456a-aa8d-7d1238acbf2e admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:47.800567 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1107/2193] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:05:47 2022] PUT /identity/v3/projects/0a1a5d620d604e468fcf86e5cdd09467/users/61973960102c4341889b29825fb38940/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:05:47.805905 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b239008-3c5d-4996-a7aa-732d7e0d8f49 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:47.817021 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b239008-3c5d-4996-a7aa-732d7e0d8f49 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:47.908859 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299fa7ac-41e4-4e8b-8fc0-bc20c2cfa004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:47.909724 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fa7ac-41e4-4e8b-8fc0-bc20c2cfa004 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:47.910110 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fa7ac-41e4-4e8b-8fc0-bc20c2cfa004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:47.910494 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fa7ac-41e4-4e8b-8fc0-bc20c2cfa004 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:47.911414 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299fa7ac-41e4-4e8b-8fc0-bc20c2cfa004 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:47.914309 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299fa7ac-41e4-4e8b-8fc0-bc20c2cfa004 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:47.928626 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1087/2194] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:05:47 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:05:47.933223 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491cad0e-16a2-456a-aa8d-7d1238acbf2e admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:47.948463 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491cad0e-16a2-456a-aa8d-7d1238acbf2e admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:47.955163 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cfd340-3eb8-4d15-bead-6a6854c4d2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:47.956473 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cfd340-3eb8-4d15-bead-6a6854c4d2ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:47.956831 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cfd340-3eb8-4d15-bead-6a6854c4d2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:47.957295 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cfd340-3eb8-4d15-bead-6a6854c4d2ce admin admin] PATH_INFO: `/v3/projects/0a1a5d620d604e468fcf86e5cdd09467/users/61973960102c4341889b29825fb38940/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:47.962160 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cfd340-3eb8-4d15-bead-6a6854c4d2ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a1a5d620d604e468fcf86e5cdd09467, user_id=61973960102c4341889b29825fb38940, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:47.969088 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cfd340-3eb8-4d15-bead-6a6854c4d2ce admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:47.986582 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1108/2195] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:05:47 2022] PUT /identity/v3/projects/0a1a5d620d604e468fcf86e5cdd09467/users/61973960102c4341889b29825fb38940/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:05:48.487345 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23344bf7-e94c-4e23-b759-31634b7d87fb None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:48.488275 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23344bf7-e94c-4e23-b759-31634b7d87fb None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:48.488736 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23344bf7-e94c-4e23-b759-31634b7d87fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:48.508106 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-23344bf7-e94c-4e23-b759-31634b7d87fb None None] MFA Rules not processed for user `7af02cd6ffd0484c8f6569629cd6313c`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:05:48.509593 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-23344bf7-e94c-4e23-b759-31634b7d87fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:05:48.582862 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1088/2196] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:05:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:05:53.822389 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f8e40-378e-4c62-9f22-e03c676102ba None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:53.822907 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f8e40-378e-4c62-9f22-e03c676102ba None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:53.823023 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f8e40-378e-4c62-9f22-e03c676102ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:53.868230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-703f8e40-378e-4c62-9f22-e03c676102ba None None] MFA Rules not processed for user `61973960102c4341889b29825fb38940`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:05:53.869458 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-703f8e40-378e-4c62-9f22-e03c676102ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:05:53.939006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1109/2197] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:05:53 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:05:53.952354 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23344bf7-e94c-4e23-b759-31634b7d87fb None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:53.953722 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-23344bf7-e94c-4e23-b759-31634b7d87fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:05:54.030362 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:05:54.073303 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23344bf7-e94c-4e23-b759-31634b7d87fb None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:54.081545 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924c5ee1-f25f-4de5-9e40-9625c5beb32d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:54.082776 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924c5ee1-f25f-4de5-9e40-9625c5beb32d service nova] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:54.083071 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924c5ee1-f25f-4de5-9e40-9625c5beb32d service nova] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:54.083393 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924c5ee1-f25f-4de5-9e40-9625c5beb32d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:54.160288 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924c5ee1-f25f-4de5-9e40-9625c5beb32d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:54.161936 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924c5ee1-f25f-4de5-9e40-9625c5beb32d service nova] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:54.174138 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1089/2198] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:05:53 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:05:59.581098 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4cdcfd-5e7a-4ca4-9c3a-8d6641cf5a6d None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:59.581659 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4cdcfd-5e7a-4ca4-9c3a-8d6641cf5a6d None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:59.581659 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4cdcfd-5e7a-4ca4-9c3a-8d6641cf5a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:59.614306 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-2f4cdcfd-5e7a-4ca4-9c3a-8d6641cf5a6d None None] MFA Rules not processed for user `86619a80df8841a3ba7d8471ac38056a`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:05:59.615559 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-2f4cdcfd-5e7a-4ca4-9c3a-8d6641cf5a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:05:59.660510 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:05:59.681259 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1110/2199] 10.222.0.13 () {58 vars in 1009 bytes} [Tue Jun 7 22:05:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:05:59.693560 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924c5ee1-f25f-4de5-9e40-9625c5beb32d service nova] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:05:59.695083 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-924c5ee1-f25f-4de5-9e40-9625c5beb32d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:05:59.770736 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924c5ee1-f25f-4de5-9e40-9625c5beb32d service nova] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:05:59.776386 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbfb635-e186-4e0c-bc08-a8ba370a00f1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1261563b6d4e50a4ffeea6a32bdb01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:05:59.777041 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbfb635-e186-4e0c-bc08-a8ba370a00f1 service swift] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:05:59.777323 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbfb635-e186-4e0c-bc08-a8ba370a00f1 service swift] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:05:59.777596 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbfb635-e186-4e0c-bc08-a8ba370a00f1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:05:59.786429 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbfb635-e186-4e0c-bc08-a8ba370a00f1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:05:59.788702 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbfb635-e186-4e0c-bc08-a8ba370a00f1 service swift] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:05:59.797448 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1090/2200] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Jun 7 22:05:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 22:06:09.673571 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=6 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:06:09.793999 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=72 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:06:14.700586 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f68107-c08d-41b4-847b-21905089bdef None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:06:14.701101 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f68107-c08d-41b4-847b-21905089bdef None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:06:14.701196 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f68107-c08d-41b4-847b-21905089bdef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:06:14.720897 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-c0f68107-c08d-41b4-847b-21905089bdef None None] MFA Rules not processed for user `c44c1788959e4b339c9cd5aad4318496`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:06:14.722110 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c0f68107-c08d-41b4-847b-21905089bdef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:06:14.789137 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1111/2201] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:06:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:06:21.806653 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbfb635-e186-4e0c-bc08-a8ba370a00f1 service swift] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:06:21.808126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-7cbfb635-e186-4e0c-bc08-a8ba370a00f1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:06:21.952119 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbfb635-e186-4e0c-bc08-a8ba370a00f1 service swift] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:06:21.957925 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea87e8-fec9-4a04-bed4-4146b3aca102 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02941c6ad2d54e48b1114d9e030c66c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:06:21.958753 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea87e8-fec9-4a04-bed4-4146b3aca102 service placement] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:06:21.959785 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea87e8-fec9-4a04-bed4-4146b3aca102 service placement] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:06:21.960267 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea87e8-fec9-4a04-bed4-4146b3aca102 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:06:21.962413 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-27ea87e8-fec9-4a04-bed4-4146b3aca102 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:06:21.981992 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ea87e8-fec9-4a04-bed4-4146b3aca102 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:06:21.982638 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ea87e8-fec9-4a04-bed4-4146b3aca102 service placement] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:06:21.996421 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1091/2202] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:06:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:06:24.782166 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=25 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:06:28.606231 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdeffef-c30f-4430-98ef-583a729bdd27 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:06:28.606894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdeffef-c30f-4430-98ef-583a729bdd27 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:06:28.606894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdeffef-c30f-4430-98ef-583a729bdd27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:06:28.625990 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-4fdeffef-c30f-4430-98ef-583a729bdd27 None None] MFA Rules not processed for user `c710942802df49ec8bd630e4b225eed2`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:06:28.626848 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-4fdeffef-c30f-4430-98ef-583a729bdd27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:06:28.634349 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1112/2203] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:06:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:06:31.991563 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=41 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:06:38.620005 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:06:49.258498 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04049610-a232-4751-b28a-929f632655c2 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:06:49.259077 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04049610-a232-4751-b28a-929f632655c2 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:06:49.259077 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04049610-a232-4751-b28a-929f632655c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:06:49.279024 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-04049610-a232-4751-b28a-929f632655c2 None None] MFA Rules not processed for user `fa4fc7744b3d4e5fb6e281fcc90cef5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:06:49.280249 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-04049610-a232-4751-b28a-929f632655c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:06:49.352273 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1092/2204] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:06:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:06:59.341639 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=25 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:07:02.182657 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b786d37-1608-417e-a767-76a99f969675 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:07:02.183215 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b786d37-1608-417e-a767-76a99f969675 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:07:02.183215 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b786d37-1608-417e-a767-76a99f969675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:07:02.234081 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-5b786d37-1608-417e-a767-76a99f969675 None None] MFA Rules not processed for user `86619a80df8841a3ba7d8471ac38056a`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:07:02.235314 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-5b786d37-1608-417e-a767-76a99f969675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:07:02.256085 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1113/2205] 10.222.0.13 () {58 vars in 1009 bytes} [Tue Jun 7 22:07:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:07:02.297615 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04049610-a232-4751-b28a-929f632655c2 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:07:02.320066 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04049610-a232-4751-b28a-929f632655c2 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:07:02.325060 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11afe23-68c6-4973-868b-418e15fa1412 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1261563b6d4e50a4ffeea6a32bdb01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:07:02.325711 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11afe23-68c6-4973-868b-418e15fa1412 service swift] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:07:02.325995 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11afe23-68c6-4973-868b-418e15fa1412 service swift] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:07:02.326273 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11afe23-68c6-4973-868b-418e15fa1412 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:07:02.334892 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11afe23-68c6-4973-868b-418e15fa1412 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:07:02.336518 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11afe23-68c6-4973-868b-418e15fa1412 service swift] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:07:02.344889 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1093/2206] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Jun 7 22:07:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 22:07:12.253139 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=18 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:07:12.341495 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=10 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:07:16.261260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b786d37-1608-417e-a767-76a99f969675 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:07:16.359050 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b786d37-1608-417e-a767-76a99f969675 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:07:16.363086 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c28d040-be44-454d-8fcd-e11a0a19a481 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:07:16.363940 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28d040-be44-454d-8fcd-e11a0a19a481 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:07:16.364613 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28d040-be44-454d-8fcd-e11a0a19a481 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:07:16.365466 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28d040-be44-454d-8fcd-e11a0a19a481 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:07:16.368164 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-4c28d040-be44-454d-8fcd-e11a0a19a481 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:07:16.452909 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c28d040-be44-454d-8fcd-e11a0a19a481 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:07:16.454437 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c28d040-be44-454d-8fcd-e11a0a19a481 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:07:16.476506 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1114/2207] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:07:16 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:07:19.146592 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041bb51e-48de-4615-97ef-89d71f2f9347 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:07:19.147620 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041bb51e-48de-4615-97ef-89d71f2f9347 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:07:19.148156 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041bb51e-48de-4615-97ef-89d71f2f9347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:07:19.171183 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-041bb51e-48de-4615-97ef-89d71f2f9347 None None] MFA Rules not processed for user `86619a80df8841a3ba7d8471ac38056a`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:07:19.172506 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-041bb51e-48de-4615-97ef-89d71f2f9347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:07:19.184986 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1094/2208] 10.222.0.13 () {58 vars in 967 bytes} [Tue Jun 7 22:07:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:07:19.198066 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c28d040-be44-454d-8fcd-e11a0a19a481 service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:07:19.210702 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c28d040-be44-454d-8fcd-e11a0a19a481 service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:07:19.216432 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe542e48-c366-47c5-9761-ef0557aafe12 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1261563b6d4e50a4ffeea6a32bdb01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:07:19.217092 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe542e48-c366-47c5-9761-ef0557aafe12 service swift] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:07:19.217378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe542e48-c366-47c5-9761-ef0557aafe12 service swift] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:07:19.217722 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe542e48-c366-47c5-9761-ef0557aafe12 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:07:19.228803 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe542e48-c366-47c5-9761-ef0557aafe12 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:07:19.230421 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe542e48-c366-47c5-9761-ef0557aafe12 service swift] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:07:19.240569 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1115/2209] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Jun 7 22:07:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 22:07:20.256107 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041bb51e-48de-4615-97ef-89d71f2f9347 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:07:20.257585 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-041bb51e-48de-4615-97ef-89d71f2f9347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:07:20.270357 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041bb51e-48de-4615-97ef-89d71f2f9347 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:07:20.273951 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e35a92-f093-4070-b20b-3f6557087659 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:07:20.274595 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e35a92-f093-4070-b20b-3f6557087659 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:07:20.274879 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e35a92-f093-4070-b20b-3f6557087659 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:07:20.275157 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e35a92-f093-4070-b20b-3f6557087659 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:07:20.287724 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e35a92-f093-4070-b20b-3f6557087659 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:07:20.290487 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e35a92-f093-4070-b20b-3f6557087659 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:07:20.302954 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1095/2210] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:07:20 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:07:21.964393 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe542e48-c366-47c5-9761-ef0557aafe12 service swift] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:07:21.966207 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-fe542e48-c366-47c5-9761-ef0557aafe12 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:07:22.059022 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe542e48-c366-47c5-9761-ef0557aafe12 service swift] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:07:22.064611 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4b5f01-f009-40ba-b757-160dbd51ea21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:07:22.065054 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b5f01-f009-40ba-b757-160dbd51ea21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:07:22.065378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b5f01-f009-40ba-b757-160dbd51ea21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:07:22.065653 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b5f01-f009-40ba-b757-160dbd51ea21 admin admin] PATH_INFO: `/v3/users/7ce9ec4aea9449d3890f6dbb93f97968` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:07:22.066874 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4b5f01-f009-40ba-b757-160dbd51ea21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ce9ec4aea9449d3890f6dbb93f97968)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:07:22.069520 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4b5f01-f009-40ba-b757-160dbd51ea21 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:07:22.129253 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:07:22.141886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-6e4b5f01-f009-40ba-b757-160dbd51ea21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ce9ec4aea9449d3890f6dbb93f97968'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:07:22.162447 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-6e4b5f01-f009-40ba-b757-160dbd51ea21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ce9ec4aea9449d3890f6dbb93f97968'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:07:22.179880 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-6e4b5f01-f009-40ba-b757-160dbd51ea21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ce9ec4aea9449d3890f6dbb93f97968'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:07:22.180864 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-6e4b5f01-f009-40ba-b757-160dbd51ea21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ce9ec4aea9449d3890f6dbb93f97968'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:07:22.181735 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-6e4b5f01-f009-40ba-b757-160dbd51ea21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ce9ec4aea9449d3890f6dbb93f97968'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:07:22.200646 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1116/2211] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:07:21 2022] DELETE /identity/v3/users/7ce9ec4aea9449d3890f6dbb93f97968 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:07:22.449047 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e35a92-f093-4070-b20b-3f6557087659 service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:07:22.460257 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e35a92-f093-4070-b20b-3f6557087659 service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:07:22.523468 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58812fb8-a99f-46f8-9cd1-5641823e6ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:07:22.524335 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58812fb8-a99f-46f8-9cd1-5641823e6ee2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:07:22.524618 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58812fb8-a99f-46f8-9cd1-5641823e6ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:07:22.524895 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58812fb8-a99f-46f8-9cd1-5641823e6ee2 admin admin] PATH_INFO: `/v3/projects/bc558dd3848d4d3d950d7cd408e19741` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:07:22.526127 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58812fb8-a99f-46f8-9cd1-5641823e6ee2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc558dd3848d4d3d950d7cd408e19741)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:07:22.528069 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58812fb8-a99f-46f8-9cd1-5641823e6ee2 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:07:22.576722 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-58812fb8-a99f-46f8-9cd1-5641823e6ee2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc558dd3848d4d3d950d7cd408e19741', 'request_id': 'req-58812fb8-a99f-46f8-9cd1-5641823e6ee2'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:07:22.584410 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-58812fb8-a99f-46f8-9cd1-5641823e6ee2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc558dd3848d4d3d950d7cd408e19741', 'request_id': 'req-58812fb8-a99f-46f8-9cd1-5641823e6ee2'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:07:22.591186 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-58812fb8-a99f-46f8-9cd1-5641823e6ee2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc558dd3848d4d3d950d7cd408e19741', 'request_id': 'req-58812fb8-a99f-46f8-9cd1-5641823e6ee2'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:07:22.602610 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-58812fb8-a99f-46f8-9cd1-5641823e6ee2 admin admin] The token cache is being invalidate because project bc558dd3848d4d3d950d7cd408e19741 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:07:22.602898 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-58812fb8-a99f-46f8-9cd1-5641823e6ee2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:07:22.603579 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-58812fb8-a99f-46f8-9cd1-5641823e6ee2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:07:22.605131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1096/2212] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:07:22 2022] DELETE /identity/v3/projects/bc558dd3848d4d3d950d7cd408e19741 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:07:22.610059 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4039f136-dfdc-4ae4-b12c-a47a888dec7b None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:07:22.610426 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4039f136-dfdc-4ae4-b12c-a47a888dec7b None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:07:22.610779 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4039f136-dfdc-4ae4-b12c-a47a888dec7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:07:22.629337 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-4039f136-dfdc-4ae4-b12c-a47a888dec7b None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:07:22.630709 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-4039f136-dfdc-4ae4-b12c-a47a888dec7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:07:22.692071 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1117/2213] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:07:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:07:22.699919 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6969b9-b5e2-411f-b928-52edefd8c5e3 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:07:22.700341 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6969b9-b5e2-411f-b928-52edefd8c5e3 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:07:22.700706 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6969b9-b5e2-411f-b928-52edefd8c5e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:07:22.725293 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-ad6969b9-b5e2-411f-b928-52edefd8c5e3 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:07:22.728357 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-ad6969b9-b5e2-411f-b928-52edefd8c5e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:07:22.741984 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1097/2214] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:07:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:07:22.746313 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4039f136-dfdc-4ae4-b12c-a47a888dec7b None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:07:22.787915 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4039f136-dfdc-4ae4-b12c-a47a888dec7b None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:07:22.794322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d88067-8d64-4c25-973e-f97a49526c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:07:22.795192 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d88067-8d64-4c25-973e-f97a49526c0e admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:07:22.795732 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d88067-8d64-4c25-973e-f97a49526c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:07:22.796136 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d88067-8d64-4c25-973e-f97a49526c0e admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:07:22.797040 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d88067-8d64-4c25-973e-f97a49526c0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:07:22.797499 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d88067-8d64-4c25-973e-f97a49526c0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:07:22.799266 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d88067-8d64-4c25-973e-f97a49526c0e admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:07:22.824379 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1118/2215] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:07:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:07:22.829369 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6969b9-b5e2-411f-b928-52edefd8c5e3 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:07:22.845830 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6969b9-b5e2-411f-b928-52edefd8c5e3 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:07:22.854695 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db9921-6756-449c-a04e-481da851ae81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:07:22.856618 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db9921-6756-449c-a04e-481da851ae81 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:07:22.857188 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db9921-6756-449c-a04e-481da851ae81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:07:22.857623 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db9921-6756-449c-a04e-481da851ae81 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:07:22.858471 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42db9921-6756-449c-a04e-481da851ae81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:07:22.860641 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42db9921-6756-449c-a04e-481da851ae81 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:07:22.898957 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1098/2216] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:07:22 2022] POST /identity/v3/projects => generated 369 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:07:22.902959 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d88067-8d64-4c25-973e-f97a49526c0e admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:07:22.917464 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d88067-8d64-4c25-973e-f97a49526c0e admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:07:22.992645 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c50dfd-2410-40c8-a37b-87e8344c32bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:07:22.993408 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c50dfd-2410-40c8-a37b-87e8344c32bd admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:07:22.994024 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c50dfd-2410-40c8-a37b-87e8344c32bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:07:22.994024 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c50dfd-2410-40c8-a37b-87e8344c32bd admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:07:22.994987 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c50dfd-2410-40c8-a37b-87e8344c32bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:07:22.996903 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c50dfd-2410-40c8-a37b-87e8344c32bd admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:07:23.020959 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1119/2217] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:07:22 2022] POST /identity/v3/users => generated 285 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:07:23.025618 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db9921-6756-449c-a04e-481da851ae81 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:07:23.040784 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db9921-6756-449c-a04e-481da851ae81 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:07:23.046243 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78eeb2a-11c2-4972-af7c-4a4eb40b35e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:07:23.046992 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78eeb2a-11c2-4972-af7c-4a4eb40b35e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:07:23.047354 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78eeb2a-11c2-4972-af7c-4a4eb40b35e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:07:23.047708 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78eeb2a-11c2-4972-af7c-4a4eb40b35e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:07:23.048528 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78eeb2a-11c2-4972-af7c-4a4eb40b35e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:07:23.050206 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78eeb2a-11c2-4972-af7c-4a4eb40b35e6 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:07:23.061569 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1099/2218] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:07:23 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:07:23.065506 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c50dfd-2410-40c8-a37b-87e8344c32bd admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:07:23.075847 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c50dfd-2410-40c8-a37b-87e8344c32bd admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:07:23.081371 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f3dd78-2c87-466f-b1b2-4e8e3ab3c98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:07:23.081974 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f3dd78-2c87-466f-b1b2-4e8e3ab3c98a admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:07:23.082262 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f3dd78-2c87-466f-b1b2-4e8e3ab3c98a admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:07:23.082590 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f3dd78-2c87-466f-b1b2-4e8e3ab3c98a admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:07:23.083407 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f3dd78-2c87-466f-b1b2-4e8e3ab3c98a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:07:23.085169 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f3dd78-2c87-466f-b1b2-4e8e3ab3c98a admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:07:23.095933 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1120/2219] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:07:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 22:07:23.100875 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78eeb2a-11c2-4972-af7c-4a4eb40b35e6 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:07:23.111615 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78eeb2a-11c2-4972-af7c-4a4eb40b35e6 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:07:23.117127 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e815ad29-53a9-4f57-93bf-3812847e6927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:07:23.118046 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815ad29-53a9-4f57-93bf-3812847e6927 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:07:23.118353 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815ad29-53a9-4f57-93bf-3812847e6927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:07:23.118634 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815ad29-53a9-4f57-93bf-3812847e6927 admin admin] PATH_INFO: `/v3/projects/33853f7eb671431992975e7bc3574241/users/2a8eed7dee294efb8c3109c990755aa6/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:07:23.134732 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e815ad29-53a9-4f57-93bf-3812847e6927 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33853f7eb671431992975e7bc3574241, user_id=2a8eed7dee294efb8c3109c990755aa6, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:07:23.139596 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e815ad29-53a9-4f57-93bf-3812847e6927 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:07:23.153958 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1100/2220] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:07:23 2022] PUT /identity/v3/projects/33853f7eb671431992975e7bc3574241/users/2a8eed7dee294efb8c3109c990755aa6/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:07:23.224260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e7adc-c61f-4285-aa72-c214cea88cd7 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:07:23.224613 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e7adc-c61f-4285-aa72-c214cea88cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:07:23.224897 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e7adc-c61f-4285-aa72-c214cea88cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:07:23.253293 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-0b9e7adc-c61f-4285-aa72-c214cea88cd7 None None] MFA Rules not processed for user `2a8eed7dee294efb8c3109c990755aa6`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:07:23.254655 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-0b9e7adc-c61f-4285-aa72-c214cea88cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:07:23.265482 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:07:23.327840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1121/2221] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:07:23 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:07:23.339157 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e815ad29-53a9-4f57-93bf-3812847e6927 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:07:23.340920 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-e815ad29-53a9-4f57-93bf-3812847e6927 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:07:23.351978 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:07:23.430972 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e815ad29-53a9-4f57-93bf-3812847e6927 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:07:23.434809 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bb5856-79ce-4489-9f89-3157397abc51 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96987b04471944bc86ab44ff9936c732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:07:23.435528 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb5856-79ce-4489-9f89-3157397abc51 service glance] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:07:23.435819 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb5856-79ce-4489-9f89-3157397abc51 service glance] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:07:23.436103 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb5856-79ce-4489-9f89-3157397abc51 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:07:23.445518 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bb5856-79ce-4489-9f89-3157397abc51 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:07:23.447047 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bb5856-79ce-4489-9f89-3157397abc51 service glance] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:07:23.458091 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1101/2222] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:07:23 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:07:33.324862 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=22 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:07:33.452315 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=32 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:07:55.482566 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269bdf84-299a-4a3e-8b65-be1aeb4639e7 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:07:55.483058 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269bdf84-299a-4a3e-8b65-be1aeb4639e7 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:07:55.483300 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269bdf84-299a-4a3e-8b65-be1aeb4639e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:07:55.501255 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-269bdf84-299a-4a3e-8b65-be1aeb4639e7 None None] MFA Rules not processed for user `607b70b4d6e14c95ae2880dd3577c8ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:07:55.502460 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-269bdf84-299a-4a3e-8b65-be1aeb4639e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:07:55.557370 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1122/2223] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:07:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:08:04.495231 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bb5856-79ce-4489-9f89-3157397abc51 service glance] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:08:04.496731 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-41bb5856-79ce-4489-9f89-3157397abc51 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:08:04.573086 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bb5856-79ce-4489-9f89-3157397abc51 service glance] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:08:04.577035 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05b94ef-0f86-4b84-ab9f-22b759501a14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:08:04.578000 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b94ef-0f86-4b84-ab9f-22b759501a14 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:08:04.578372 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b94ef-0f86-4b84-ab9f-22b759501a14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:08:04.579135 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b94ef-0f86-4b84-ab9f-22b759501a14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:08:04.581007 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-a05b94ef-0f86-4b84-ab9f-22b759501a14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:08:04.662528 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05b94ef-0f86-4b84-ab9f-22b759501a14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:08:04.664558 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05b94ef-0f86-4b84-ab9f-22b759501a14 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:08:04.675312 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1102/2224] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:08:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:08:05.549583 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=25 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:08:05.844934 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269bdf84-299a-4a3e-8b65-be1aeb4639e7 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:08:05.846322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-269bdf84-299a-4a3e-8b65-be1aeb4639e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:08:05.859305 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269bdf84-299a-4a3e-8b65-be1aeb4639e7 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:08:05.862824 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f069b-c92b-4589-8061-bf22ddcb88d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:08:05.863496 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f069b-c92b-4589-8061-bf22ddcb88d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:08:05.863775 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f069b-c92b-4589-8061-bf22ddcb88d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:08:05.864049 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f069b-c92b-4589-8061-bf22ddcb88d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:08:05.865756 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-332f069b-c92b-4589-8061-bf22ddcb88d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:08:05.880124 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332f069b-c92b-4589-8061-bf22ddcb88d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:08:05.881484 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332f069b-c92b-4589-8061-bf22ddcb88d3 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:08:05.892120 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1123/2225] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:08:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:08:14.670508 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=54 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:08:15.887465 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=6 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:08:22.815838 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05b94ef-0f86-4b84-ab9f-22b759501a14 service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:08:22.825150 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05b94ef-0f86-4b84-ab9f-22b759501a14 service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:08:22.829176 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a8d605-a030-4ddb-9387-b690b175e7b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:08:22.829834 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a8d605-a030-4ddb-9387-b690b175e7b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:08:22.830115 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a8d605-a030-4ddb-9387-b690b175e7b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:08:22.830389 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a8d605-a030-4ddb-9387-b690b175e7b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:08:22.832210 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-07a8d605-a030-4ddb-9387-b690b175e7b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:08:22.841990 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a8d605-a030-4ddb-9387-b690b175e7b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:08:22.843436 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a8d605-a030-4ddb-9387-b690b175e7b8 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:08:22.851907 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1103/2226] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:08:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:08:24.300561 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f069b-c92b-4589-8061-bf22ddcb88d3 service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:08:24.362209 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f069b-c92b-4589-8061-bf22ddcb88d3 service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:08:24.368497 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3d8b18-2b59-4d48-b26f-55b41c1c4826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:08:24.369445 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d8b18-2b59-4d48-b26f-55b41c1c4826 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:08:24.369790 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d8b18-2b59-4d48-b26f-55b41c1c4826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:08:24.370209 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d8b18-2b59-4d48-b26f-55b41c1c4826 admin admin] PATH_INFO: `/v3/users/2a8eed7dee294efb8c3109c990755aa6` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:08:24.371557 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3d8b18-2b59-4d48-b26f-55b41c1c4826 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a8eed7dee294efb8c3109c990755aa6)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:08:24.373480 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3d8b18-2b59-4d48-b26f-55b41c1c4826 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:08:24.420390 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3f3d8b18-2b59-4d48-b26f-55b41c1c4826 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a8eed7dee294efb8c3109c990755aa6'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:08:24.427157 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3f3d8b18-2b59-4d48-b26f-55b41c1c4826 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a8eed7dee294efb8c3109c990755aa6'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:08:24.439249 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3f3d8b18-2b59-4d48-b26f-55b41c1c4826 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a8eed7dee294efb8c3109c990755aa6'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:08:24.439968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3f3d8b18-2b59-4d48-b26f-55b41c1c4826 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a8eed7dee294efb8c3109c990755aa6'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:08:24.440565 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3f3d8b18-2b59-4d48-b26f-55b41c1c4826 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a8eed7dee294efb8c3109c990755aa6'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:08:24.454229 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1124/2227] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:08:24 2022] DELETE /identity/v3/users/2a8eed7dee294efb8c3109c990755aa6 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:08:24.458708 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b81b6-e417-4c3e-9f32-605990097287 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:08:24.459235 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b81b6-e417-4c3e-9f32-605990097287 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:08:24.459362 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b81b6-e417-4c3e-9f32-605990097287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:08:24.477492 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-ae9b81b6-e417-4c3e-9f32-605990097287 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:08:24.478693 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-ae9b81b6-e417-4c3e-9f32-605990097287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:08:24.544792 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1104/2228] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:08:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:08:24.555031 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3d8b18-2b59-4d48-b26f-55b41c1c4826 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:08:24.556490 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-3f3d8b18-2b59-4d48-b26f-55b41c1c4826 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:08:24.620221 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3d8b18-2b59-4d48-b26f-55b41c1c4826 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:08:24.623943 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a70255-a22d-4b8f-800d-fb42fb5be9a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:08:24.624627 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a70255-a22d-4b8f-800d-fb42fb5be9a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:08:24.624920 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a70255-a22d-4b8f-800d-fb42fb5be9a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:08:24.625197 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a70255-a22d-4b8f-800d-fb42fb5be9a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:08:24.634807 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a70255-a22d-4b8f-800d-fb42fb5be9a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:08:24.636382 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a70255-a22d-4b8f-800d-fb42fb5be9a2 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:08:24.646294 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1125/2229] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:08:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:08:24.912888 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9b81b6-e417-4c3e-9f32-605990097287 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:08:24.915690 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-ae9b81b6-e417-4c3e-9f32-605990097287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:08:24.932561 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9b81b6-e417-4c3e-9f32-605990097287 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:08:24.937343 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e0994-83cc-46b7-aaa5-35cddad5d6c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:08:24.938222 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e0994-83cc-46b7-aaa5-35cddad5d6c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:08:24.938504 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e0994-83cc-46b7-aaa5-35cddad5d6c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:08:24.938827 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e0994-83cc-46b7-aaa5-35cddad5d6c8 admin admin] PATH_INFO: `/v3/projects/33853f7eb671431992975e7bc3574241` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:08:24.940313 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507e0994-83cc-46b7-aaa5-35cddad5d6c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33853f7eb671431992975e7bc3574241)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:08:24.942030 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507e0994-83cc-46b7-aaa5-35cddad5d6c8 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:08:24.992237 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-507e0994-83cc-46b7-aaa5-35cddad5d6c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33853f7eb671431992975e7bc3574241', 'request_id': 'req-507e0994-83cc-46b7-aaa5-35cddad5d6c8'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:08:24.999473 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-507e0994-83cc-46b7-aaa5-35cddad5d6c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33853f7eb671431992975e7bc3574241', 'request_id': 'req-507e0994-83cc-46b7-aaa5-35cddad5d6c8'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:08:25.005817 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-507e0994-83cc-46b7-aaa5-35cddad5d6c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33853f7eb671431992975e7bc3574241', 'request_id': 'req-507e0994-83cc-46b7-aaa5-35cddad5d6c8'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:08:25.016087 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-507e0994-83cc-46b7-aaa5-35cddad5d6c8 admin admin] The token cache is being invalidate because project 33853f7eb671431992975e7bc3574241 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:08:25.016374 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-507e0994-83cc-46b7-aaa5-35cddad5d6c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:08:25.016999 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-507e0994-83cc-46b7-aaa5-35cddad5d6c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:08:25.018528 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1105/2230] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:08:24 2022] DELETE /identity/v3/projects/33853f7eb671431992975e7bc3574241 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:08:25.024496 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e892e6f0-0dad-4531-8a7f-8bc84c2db30f None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:08:25.024788 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e892e6f0-0dad-4531-8a7f-8bc84c2db30f None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:08:25.025069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e892e6f0-0dad-4531-8a7f-8bc84c2db30f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:08:25.043611 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-e892e6f0-0dad-4531-8a7f-8bc84c2db30f None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:08:25.044836 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-e892e6f0-0dad-4531-8a7f-8bc84c2db30f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:08:25.061842 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:08:25.105874 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1126/2231] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:08:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:08:25.113530 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d875d2-dfb0-4f6e-b540-adeef4d6e7f0 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:08:25.113826 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d875d2-dfb0-4f6e-b540-adeef4d6e7f0 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:08:25.114111 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d875d2-dfb0-4f6e-b540-adeef4d6e7f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:08:25.132222 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-83d875d2-dfb0-4f6e-b540-adeef4d6e7f0 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:08:25.133537 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-83d875d2-dfb0-4f6e-b540-adeef4d6e7f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:08:25.141349 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1106/2232] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:08:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:08:25.145614 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e892e6f0-0dad-4531-8a7f-8bc84c2db30f None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:08:25.156262 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e892e6f0-0dad-4531-8a7f-8bc84c2db30f None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:08:25.160848 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94ef846-6552-42a8-9b5e-7db04bfaf256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:08:25.161496 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ef846-6552-42a8-9b5e-7db04bfaf256 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:08:25.161787 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ef846-6552-42a8-9b5e-7db04bfaf256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:08:25.162066 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ef846-6552-42a8-9b5e-7db04bfaf256 admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:08:25.162788 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94ef846-6552-42a8-9b5e-7db04bfaf256 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:08:25.163095 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94ef846-6552-42a8-9b5e-7db04bfaf256 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:08:25.165341 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94ef846-6552-42a8-9b5e-7db04bfaf256 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:08:25.176701 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1127/2233] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:08:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:08:25.180917 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d875d2-dfb0-4f6e-b540-adeef4d6e7f0 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:08:25.190020 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d875d2-dfb0-4f6e-b540-adeef4d6e7f0 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:08:25.195249 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c67e3a-af76-411f-b56c-589ace1f592f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:08:25.195907 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c67e3a-af76-411f-b56c-589ace1f592f admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:08:25.196193 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c67e3a-af76-411f-b56c-589ace1f592f admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:08:25.196472 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c67e3a-af76-411f-b56c-589ace1f592f admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:08:25.197157 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c67e3a-af76-411f-b56c-589ace1f592f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:08:25.199045 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c67e3a-af76-411f-b56c-589ace1f592f admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:08:25.224844 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1107/2234] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:08:25 2022] POST /identity/v3/projects => generated 375 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:08:25.229472 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94ef846-6552-42a8-9b5e-7db04bfaf256 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:08:25.239894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94ef846-6552-42a8-9b5e-7db04bfaf256 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:08:25.294788 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dd9115-7347-46f0-8b45-cbaaabf249d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:08:25.295474 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd9115-7347-46f0-8b45-cbaaabf249d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:08:25.295757 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd9115-7347-46f0-8b45-cbaaabf249d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:08:25.296031 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd9115-7347-46f0-8b45-cbaaabf249d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:08:25.296723 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dd9115-7347-46f0-8b45-cbaaabf249d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:08:25.298626 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dd9115-7347-46f0-8b45-cbaaabf249d5 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:08:25.321441 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1128/2235] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:08:25 2022] POST /identity/v3/users => generated 288 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:08:25.323047 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c67e3a-af76-411f-b56c-589ace1f592f admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:08:25.333141 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c67e3a-af76-411f-b56c-589ace1f592f admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:08:25.338270 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a85492-feb2-4346-a781-8ca8c09b5b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:08:25.338919 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a85492-feb2-4346-a781-8ca8c09b5b87 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:08:25.339202 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a85492-feb2-4346-a781-8ca8c09b5b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:08:25.339519 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a85492-feb2-4346-a781-8ca8c09b5b87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:08:25.340242 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a85492-feb2-4346-a781-8ca8c09b5b87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:08:25.341828 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a85492-feb2-4346-a781-8ca8c09b5b87 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:08:25.352462 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1108/2236] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:08:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 22:08:25.356456 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dd9115-7347-46f0-8b45-cbaaabf249d5 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:08:25.366277 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dd9115-7347-46f0-8b45-cbaaabf249d5 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:08:25.371469 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beda91a5-c14b-4aac-9f83-36cc5f71ffe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:08:25.372132 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda91a5-c14b-4aac-9f83-36cc5f71ffe5 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:08:25.372450 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda91a5-c14b-4aac-9f83-36cc5f71ffe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:08:25.372759 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda91a5-c14b-4aac-9f83-36cc5f71ffe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:08:25.373510 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beda91a5-c14b-4aac-9f83-36cc5f71ffe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:08:25.375142 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beda91a5-c14b-4aac-9f83-36cc5f71ffe5 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:08:25.386091 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1129/2237] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:08:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:08:25.389931 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a85492-feb2-4346-a781-8ca8c09b5b87 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:08:25.400365 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a85492-feb2-4346-a781-8ca8c09b5b87 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:08:25.405044 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9de5acf-0fa0-45d9-b544-ea83f858f280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:08:25.405907 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de5acf-0fa0-45d9-b544-ea83f858f280 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:08:25.406187 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de5acf-0fa0-45d9-b544-ea83f858f280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:08:25.406463 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de5acf-0fa0-45d9-b544-ea83f858f280 admin admin] PATH_INFO: `/v3/projects/d07d2383b309471bbc24c6cda900d828/users/f9ad6df3072245c1b98d71d00c9f61f5/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:08:25.420700 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9de5acf-0fa0-45d9-b544-ea83f858f280 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d07d2383b309471bbc24c6cda900d828, user_id=f9ad6df3072245c1b98d71d00c9f61f5, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:08:25.423346 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9de5acf-0fa0-45d9-b544-ea83f858f280 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:08:25.436580 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1109/2238] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:08:25 2022] PUT /identity/v3/projects/d07d2383b309471bbc24c6cda900d828/users/f9ad6df3072245c1b98d71d00c9f61f5/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:08:25.448256 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e2e7f3-f2f3-4d63-9785-449e40c5a9f0 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:08:25.448542 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e2e7f3-f2f3-4d63-9785-449e40c5a9f0 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:08:25.448816 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e2e7f3-f2f3-4d63-9785-449e40c5a9f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:08:25.476267 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-d0e2e7f3-f2f3-4d63-9785-449e40c5a9f0 None None] MFA Rules not processed for user `f9ad6df3072245c1b98d71d00c9f61f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:08:25.477739 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d0e2e7f3-f2f3-4d63-9785-449e40c5a9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:08:25.542728 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1130/2239] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:08:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:08:25.553512 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9de5acf-0fa0-45d9-b544-ea83f858f280 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:08:25.555780 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-c9de5acf-0fa0-45d9-b544-ea83f858f280 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:08:25.565992 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:08:25.640564 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9de5acf-0fa0-45d9-b544-ea83f858f280 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:08:25.644156 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d5af8-e5ea-43bf-9b6a-0c02e99f913a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:08:25.644799 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d5af8-e5ea-43bf-9b6a-0c02e99f913a service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:08:25.645079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d5af8-e5ea-43bf-9b6a-0c02e99f913a service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:08:25.645376 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d5af8-e5ea-43bf-9b6a-0c02e99f913a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:08:25.653533 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6d5af8-e5ea-43bf-9b6a-0c02e99f913a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:08:25.655058 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6d5af8-e5ea-43bf-9b6a-0c02e99f913a service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:08:25.664189 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1110/2240] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:08:25 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:08:26.843111 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e2e7f3-f2f3-4d63-9785-449e40c5a9f0 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:08:26.844576 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d0e2e7f3-f2f3-4d63-9785-449e40c5a9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:08:26.864472 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:08:26.917919 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e2e7f3-f2f3-4d63-9785-449e40c5a9f0 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:08:26.923483 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3978013a-7f96-4c38-95f4-00fbd1f03094 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:08:26.924115 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3978013a-7f96-4c38-95f4-00fbd1f03094 service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:08:26.924394 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3978013a-7f96-4c38-95f4-00fbd1f03094 service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:08:26.924669 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3978013a-7f96-4c38-95f4-00fbd1f03094 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:08:26.926396 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-3978013a-7f96-4c38-95f4-00fbd1f03094 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:08:26.984070 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3978013a-7f96-4c38-95f4-00fbd1f03094 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:08:26.985700 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3978013a-7f96-4c38-95f4-00fbd1f03094 service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:08:26.994831 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1131/2241] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:08:26 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:08:35.659981 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=32 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:08:36.990593 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=48 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:08:42.932588 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d5af8-e5ea-43bf-9b6a-0c02e99f913a service cinder] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:08:43.014182 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d5af8-e5ea-43bf-9b6a-0c02e99f913a service cinder] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:08:43.019647 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8465407-0c45-4f7d-b759-803d3b281210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:08:43.020885 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8465407-0c45-4f7d-b759-803d3b281210 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:08:43.021267 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8465407-0c45-4f7d-b759-803d3b281210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:08:43.021606 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8465407-0c45-4f7d-b759-803d3b281210 admin admin] PATH_INFO: `/v3/users/f9ad6df3072245c1b98d71d00c9f61f5` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:08:43.023163 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8465407-0c45-4f7d-b759-803d3b281210 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9ad6df3072245c1b98d71d00c9f61f5)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:08:43.025378 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8465407-0c45-4f7d-b759-803d3b281210 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:08:43.083943 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-a8465407-0c45-4f7d-b759-803d3b281210 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9ad6df3072245c1b98d71d00c9f61f5'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:08:43.084708 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-a8465407-0c45-4f7d-b759-803d3b281210 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9ad6df3072245c1b98d71d00c9f61f5'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:08:43.094657 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-a8465407-0c45-4f7d-b759-803d3b281210 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9ad6df3072245c1b98d71d00c9f61f5'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:08:43.104812 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-a8465407-0c45-4f7d-b759-803d3b281210 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9ad6df3072245c1b98d71d00c9f61f5'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:08:43.112494 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-a8465407-0c45-4f7d-b759-803d3b281210 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9ad6df3072245c1b98d71d00c9f61f5'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:08:43.118970 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1111/2242] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:08:42 2022] DELETE /identity/v3/users/f9ad6df3072245c1b98d71d00c9f61f5 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:08:43.124465 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f32e4d-f62f-4c05-85ed-7172f7ea0a7a None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:08:43.124952 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f32e4d-f62f-4c05-85ed-7172f7ea0a7a None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:08:43.125153 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f32e4d-f62f-4c05-85ed-7172f7ea0a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:08:43.145370 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-66f32e4d-f62f-4c05-85ed-7172f7ea0a7a None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:08:43.146687 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-66f32e4d-f62f-4c05-85ed-7172f7ea0a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:08:43.207598 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1132/2243] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:08:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:08:43.217166 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8465407-0c45-4f7d-b759-803d3b281210 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:08:43.218703 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-a8465407-0c45-4f7d-b759-803d3b281210 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:08:43.275950 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8465407-0c45-4f7d-b759-803d3b281210 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:08:43.279757 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7d8874-37e6-4dc6-8b2b-09fe0d23f029 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:08:43.280384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d8874-37e6-4dc6-8b2b-09fe0d23f029 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:08:43.280671 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d8874-37e6-4dc6-8b2b-09fe0d23f029 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:08:43.280964 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d8874-37e6-4dc6-8b2b-09fe0d23f029 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:08:43.290063 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7d8874-37e6-4dc6-8b2b-09fe0d23f029 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:08:43.291540 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7d8874-37e6-4dc6-8b2b-09fe0d23f029 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:08:43.301448 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1112/2244] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:08:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:08:43.576263 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f32e4d-f62f-4c05-85ed-7172f7ea0a7a None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:08:43.577889 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-66f32e4d-f62f-4c05-85ed-7172f7ea0a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:08:43.593501 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f32e4d-f62f-4c05-85ed-7172f7ea0a7a None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:08:43.599227 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b8a6ca-2fd4-4ade-afd0-90deb2626574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:08:43.600101 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b8a6ca-2fd4-4ade-afd0-90deb2626574 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:08:43.600381 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b8a6ca-2fd4-4ade-afd0-90deb2626574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:08:43.600701 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b8a6ca-2fd4-4ade-afd0-90deb2626574 admin admin] PATH_INFO: `/v3/projects/d07d2383b309471bbc24c6cda900d828` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:08:43.602144 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b8a6ca-2fd4-4ade-afd0-90deb2626574 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d07d2383b309471bbc24c6cda900d828)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:08:43.604138 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b8a6ca-2fd4-4ade-afd0-90deb2626574 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:08:43.650969 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-64b8a6ca-2fd4-4ade-afd0-90deb2626574 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd07d2383b309471bbc24c6cda900d828', 'request_id': 'req-64b8a6ca-2fd4-4ade-afd0-90deb2626574'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:08:43.659056 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-64b8a6ca-2fd4-4ade-afd0-90deb2626574 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd07d2383b309471bbc24c6cda900d828', 'request_id': 'req-64b8a6ca-2fd4-4ade-afd0-90deb2626574'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:08:43.664867 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-64b8a6ca-2fd4-4ade-afd0-90deb2626574 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd07d2383b309471bbc24c6cda900d828', 'request_id': 'req-64b8a6ca-2fd4-4ade-afd0-90deb2626574'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:08:43.676596 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-64b8a6ca-2fd4-4ade-afd0-90deb2626574 admin admin] The token cache is being invalidate because project d07d2383b309471bbc24c6cda900d828 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:08:43.676886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-64b8a6ca-2fd4-4ade-afd0-90deb2626574 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:08:43.677531 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-64b8a6ca-2fd4-4ade-afd0-90deb2626574 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:08:43.679069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1133/2245] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:08:43 2022] DELETE /identity/v3/projects/d07d2383b309471bbc24c6cda900d828 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:08:49.277652 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7d8874-37e6-4dc6-8b2b-09fe0d23f029 service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:08:49.280050 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-4c7d8874-37e6-4dc6-8b2b-09fe0d23f029 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:08:49.292771 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7d8874-37e6-4dc6-8b2b-09fe0d23f029 service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:08:49.311038 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:08:49.351401 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9467134-4318-491d-a7a1-7b584bf47bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:08:49.352536 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9467134-4318-491d-a7a1-7b584bf47bb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:08:49.352816 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9467134-4318-491d-a7a1-7b584bf47bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:08:49.353092 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9467134-4318-491d-a7a1-7b584bf47bb8 admin admin] PATH_INFO: `/v3/users/fa4fc7744b3d4e5fb6e281fcc90cef5e` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:08:49.354282 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9467134-4318-491d-a7a1-7b584bf47bb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa4fc7744b3d4e5fb6e281fcc90cef5e)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:08:49.356217 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9467134-4318-491d-a7a1-7b584bf47bb8 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:08:49.401873 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c9467134-4318-491d-a7a1-7b584bf47bb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa4fc7744b3d4e5fb6e281fcc90cef5e'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:08:49.402523 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c9467134-4318-491d-a7a1-7b584bf47bb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa4fc7744b3d4e5fb6e281fcc90cef5e'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:08:49.410507 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c9467134-4318-491d-a7a1-7b584bf47bb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa4fc7744b3d4e5fb6e281fcc90cef5e'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:08:49.422238 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c9467134-4318-491d-a7a1-7b584bf47bb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa4fc7744b3d4e5fb6e281fcc90cef5e'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:08:49.429025 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c9467134-4318-491d-a7a1-7b584bf47bb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa4fc7744b3d4e5fb6e281fcc90cef5e'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:08:49.436302 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1113/2246] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:08:49 2022] DELETE /identity/v3/users/fa4fc7744b3d4e5fb6e281fcc90cef5e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:08:49.612330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b8a6ca-2fd4-4ade-afd0-90deb2626574 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:08:49.613751 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-64b8a6ca-2fd4-4ade-afd0-90deb2626574 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:08:49.627683 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b8a6ca-2fd4-4ade-afd0-90deb2626574 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:08:49.685249 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecac6c2-572f-43a0-a4b0-fcb1bab01db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:08:49.686119 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecac6c2-572f-43a0-a4b0-fcb1bab01db9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:08:49.686406 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecac6c2-572f-43a0-a4b0-fcb1bab01db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:08:49.686683 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecac6c2-572f-43a0-a4b0-fcb1bab01db9 admin admin] PATH_INFO: `/v3/projects/d3b84dd581fc4e9499cfecbb7242e3b3` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:08:49.687921 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecac6c2-572f-43a0-a4b0-fcb1bab01db9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3b84dd581fc4e9499cfecbb7242e3b3)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:08:49.689837 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecac6c2-572f-43a0-a4b0-fcb1bab01db9 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:08:49.719683 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:08:49.737510 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-2ecac6c2-572f-43a0-a4b0-fcb1bab01db9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3b84dd581fc4e9499cfecbb7242e3b3', 'request_id': 'req-2ecac6c2-572f-43a0-a4b0-fcb1bab01db9'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:08:49.744767 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-2ecac6c2-572f-43a0-a4b0-fcb1bab01db9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3b84dd581fc4e9499cfecbb7242e3b3', 'request_id': 'req-2ecac6c2-572f-43a0-a4b0-fcb1bab01db9'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:08:49.751447 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-2ecac6c2-572f-43a0-a4b0-fcb1bab01db9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3b84dd581fc4e9499cfecbb7242e3b3', 'request_id': 'req-2ecac6c2-572f-43a0-a4b0-fcb1bab01db9'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:08:49.767739 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-2ecac6c2-572f-43a0-a4b0-fcb1bab01db9 admin admin] The token cache is being invalidate because project d3b84dd581fc4e9499cfecbb7242e3b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:08:49.768329 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-2ecac6c2-572f-43a0-a4b0-fcb1bab01db9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:08:49.769016 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-2ecac6c2-572f-43a0-a4b0-fcb1bab01db9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:08:49.770827 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1134/2247] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:08:49 2022] DELETE /identity/v3/projects/d3b84dd581fc4e9499cfecbb7242e3b3 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:08:49.776890 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f016656-7ea6-440a-b88f-a278fa94cb6a None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:08:49.777645 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f016656-7ea6-440a-b88f-a278fa94cb6a None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:08:49.778101 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f016656-7ea6-440a-b88f-a278fa94cb6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:08:49.804352 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-2f016656-7ea6-440a-b88f-a278fa94cb6a None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:08:49.805660 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-2f016656-7ea6-440a-b88f-a278fa94cb6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:08:49.870420 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1114/2248] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:08:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:08:49.878950 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0c438-b77b-45ec-bf9a-84a28f26d32c None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:08:49.879246 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0c438-b77b-45ec-bf9a-84a28f26d32c None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:08:49.879563 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0c438-b77b-45ec-bf9a-84a28f26d32c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:08:49.899060 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-c8d0c438-b77b-45ec-bf9a-84a28f26d32c None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:08:49.900293 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c8d0c438-b77b-45ec-bf9a-84a28f26d32c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:08:49.908904 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1135/2249] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:08:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:08:49.913330 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f016656-7ea6-440a-b88f-a278fa94cb6a None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:08:49.922981 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f016656-7ea6-440a-b88f-a278fa94cb6a None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:08:49.927557 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1decdc-bdec-4879-a82b-65043596ceb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:08:49.928207 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1decdc-bdec-4879-a82b-65043596ceb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:08:49.928490 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1decdc-bdec-4879-a82b-65043596ceb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:08:49.928773 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1decdc-bdec-4879-a82b-65043596ceb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:08:49.929517 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1decdc-bdec-4879-a82b-65043596ceb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:08:49.929812 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1decdc-bdec-4879-a82b-65043596ceb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:08:49.931414 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1decdc-bdec-4879-a82b-65043596ceb2 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:08:49.943043 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1115/2250] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:08:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:08:49.947530 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d0c438-b77b-45ec-bf9a-84a28f26d32c None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:08:49.957871 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d0c438-b77b-45ec-bf9a-84a28f26d32c None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:08:49.962890 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d84467-7244-430e-8b9e-875d5c7eae8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:08:49.963664 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d84467-7244-430e-8b9e-875d5c7eae8f admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:08:49.964026 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d84467-7244-430e-8b9e-875d5c7eae8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:08:49.964401 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d84467-7244-430e-8b9e-875d5c7eae8f admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:08:49.965165 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d84467-7244-430e-8b9e-875d5c7eae8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:08:49.967753 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d84467-7244-430e-8b9e-875d5c7eae8f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:08:49.994909 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1136/2251] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:08:49 2022] POST /identity/v3/projects => generated 399 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:08:49.999130 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1decdc-bdec-4879-a82b-65043596ceb2 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:08:50.009040 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1decdc-bdec-4879-a82b-65043596ceb2 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:08:50.035559 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:08:50.069089 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4249d08-d352-4aeb-9a68-ceedb72bbbbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:08:50.069777 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4249d08-d352-4aeb-9a68-ceedb72bbbbd admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:08:50.070055 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4249d08-d352-4aeb-9a68-ceedb72bbbbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:08:50.070441 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4249d08-d352-4aeb-9a68-ceedb72bbbbd admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:08:50.071184 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4249d08-d352-4aeb-9a68-ceedb72bbbbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:08:50.073149 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4249d08-d352-4aeb-9a68-ceedb72bbbbd admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:08:50.096654 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1116/2252] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:08:49 2022] POST /identity/v3/users => generated 300 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:08:50.106486 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d84467-7244-430e-8b9e-875d5c7eae8f admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:08:50.119861 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d84467-7244-430e-8b9e-875d5c7eae8f admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:08:50.126400 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3062b0-259c-4f89-85bf-a64e8bdad3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:08:50.127434 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3062b0-259c-4f89-85bf-a64e8bdad3be admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:08:50.127868 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3062b0-259c-4f89-85bf-a64e8bdad3be admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:08:50.128320 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3062b0-259c-4f89-85bf-a64e8bdad3be admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:08:50.129443 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3062b0-259c-4f89-85bf-a64e8bdad3be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:08:50.132057 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3062b0-259c-4f89-85bf-a64e8bdad3be admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:08:50.145261 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1137/2253] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:08:50 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:08:50.149320 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4249d08-d352-4aeb-9a68-ceedb72bbbbd admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:08:50.160988 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4249d08-d352-4aeb-9a68-ceedb72bbbbd admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:08:50.165735 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f575e20e-f20a-4cb6-b82f-0a6493b95397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:08:50.166472 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f575e20e-f20a-4cb6-b82f-0a6493b95397 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:08:50.166825 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f575e20e-f20a-4cb6-b82f-0a6493b95397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:08:50.167172 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f575e20e-f20a-4cb6-b82f-0a6493b95397 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:08:50.168033 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f575e20e-f20a-4cb6-b82f-0a6493b95397 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:08:50.169759 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f575e20e-f20a-4cb6-b82f-0a6493b95397 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:08:50.180085 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1117/2254] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:08:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:08:50.184343 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3062b0-259c-4f89-85bf-a64e8bdad3be admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:08:50.194941 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3062b0-259c-4f89-85bf-a64e8bdad3be admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:08:50.199950 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a6ffd-0987-4c09-987e-65d20bfa2615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:08:50.201600 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a6ffd-0987-4c09-987e-65d20bfa2615 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:08:50.201885 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a6ffd-0987-4c09-987e-65d20bfa2615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:08:50.202160 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a6ffd-0987-4c09-987e-65d20bfa2615 admin admin] PATH_INFO: `/v3/projects/b3ab47ea6d88431d8d20fbbf818773b6/users/6e60c93c6765487eab8c1ac1525e83dc/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:08:50.216909 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1a6ffd-0987-4c09-987e-65d20bfa2615 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3ab47ea6d88431d8d20fbbf818773b6, user_id=6e60c93c6765487eab8c1ac1525e83dc, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:08:50.219585 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1a6ffd-0987-4c09-987e-65d20bfa2615 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:08:50.234194 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1138/2255] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:08:50 2022] PUT /identity/v3/projects/b3ab47ea6d88431d8d20fbbf818773b6/users/6e60c93c6765487eab8c1ac1525e83dc/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:08:50.244765 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8b300-519d-4ae2-9fe6-78a494dcde3e None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:08:50.245118 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8b300-519d-4ae2-9fe6-78a494dcde3e None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:08:50.245356 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8b300-519d-4ae2-9fe6-78a494dcde3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:08:50.273414 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-d9e8b300-519d-4ae2-9fe6-78a494dcde3e None None] MFA Rules not processed for user `6e60c93c6765487eab8c1ac1525e83dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:08:50.274627 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-d9e8b300-519d-4ae2-9fe6-78a494dcde3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:08:50.343571 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1118/2256] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:08:50 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:08:50.354841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a6ffd-0987-4c09-987e-65d20bfa2615 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:08:50.356658 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-5e1a6ffd-0987-4c09-987e-65d20bfa2615 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:08:50.414427 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a6ffd-0987-4c09-987e-65d20bfa2615 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:08:50.417957 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a21a3-4b01-4cd6-bccf-257a8a67704a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:08:50.418591 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a21a3-4b01-4cd6-bccf-257a8a67704a service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:08:50.418870 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a21a3-4b01-4cd6-bccf-257a8a67704a service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:08:50.419365 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a21a3-4b01-4cd6-bccf-257a8a67704a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:08:50.427733 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1a21a3-4b01-4cd6-bccf-257a8a67704a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:08:50.429389 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1a21a3-4b01-4cd6-bccf-257a8a67704a service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:08:50.439730 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1139/2257] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:08:50 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:08:59.651729 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8b300-519d-4ae2-9fe6-78a494dcde3e None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:08:59.713728 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8b300-519d-4ae2-9fe6-78a494dcde3e None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:08:59.718809 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c973bd69-1063-4014-a6dc-fabc71ee1f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:08:59.719700 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973bd69-1063-4014-a6dc-fabc71ee1f9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:08:59.719982 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973bd69-1063-4014-a6dc-fabc71ee1f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:08:59.720258 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973bd69-1063-4014-a6dc-fabc71ee1f9c admin admin] PATH_INFO: `/v3/users/6e60c93c6765487eab8c1ac1525e83dc` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:08:59.721431 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c973bd69-1063-4014-a6dc-fabc71ee1f9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e60c93c6765487eab8c1ac1525e83dc)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:08:59.723389 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c973bd69-1063-4014-a6dc-fabc71ee1f9c admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:08:59.768455 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:08:59.774528 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c973bd69-1063-4014-a6dc-fabc71ee1f9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e60c93c6765487eab8c1ac1525e83dc'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:08:59.775463 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c973bd69-1063-4014-a6dc-fabc71ee1f9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e60c93c6765487eab8c1ac1525e83dc'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:08:59.783293 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c973bd69-1063-4014-a6dc-fabc71ee1f9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e60c93c6765487eab8c1ac1525e83dc'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:08:59.795279 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c973bd69-1063-4014-a6dc-fabc71ee1f9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e60c93c6765487eab8c1ac1525e83dc'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:08:59.802060 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c973bd69-1063-4014-a6dc-fabc71ee1f9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e60c93c6765487eab8c1ac1525e83dc'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:08:59.808037 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1119/2258] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:08:59 2022] DELETE /identity/v3/users/6e60c93c6765487eab8c1ac1525e83dc => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:08:59.813161 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62835bc2-4e04-4dfe-9f7d-fe7d0e23e043 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:08:59.813665 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62835bc2-4e04-4dfe-9f7d-fe7d0e23e043 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:08:59.813777 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62835bc2-4e04-4dfe-9f7d-fe7d0e23e043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:08:59.832526 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-62835bc2-4e04-4dfe-9f7d-fe7d0e23e043 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:08:59.833745 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-62835bc2-4e04-4dfe-9f7d-fe7d0e23e043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:08:59.881589 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:08:59.895869 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1140/2259] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:08:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:08:59.905449 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c973bd69-1063-4014-a6dc-fabc71ee1f9c admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:08:59.906837 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-c973bd69-1063-4014-a6dc-fabc71ee1f9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:08:59.964268 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c973bd69-1063-4014-a6dc-fabc71ee1f9c admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:08:59.968166 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13485d52-9749-46f0-8938-8fc2b1df167a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:08:59.968829 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13485d52-9749-46f0-8938-8fc2b1df167a service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:08:59.969114 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13485d52-9749-46f0-8938-8fc2b1df167a service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:08:59.969394 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13485d52-9749-46f0-8938-8fc2b1df167a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:08:59.978641 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13485d52-9749-46f0-8938-8fc2b1df167a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:08:59.980088 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13485d52-9749-46f0-8938-8fc2b1df167a service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:08:59.989306 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1120/2260] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:08:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:09:00.250629 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62835bc2-4e04-4dfe-9f7d-fe7d0e23e043 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:09:00.252192 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-62835bc2-4e04-4dfe-9f7d-fe7d0e23e043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:09:00.265916 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62835bc2-4e04-4dfe-9f7d-fe7d0e23e043 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:09:00.270682 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19280b5f-ec2e-463d-a346-9c006dae9abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:09:00.271572 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19280b5f-ec2e-463d-a346-9c006dae9abc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:09:00.271853 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19280b5f-ec2e-463d-a346-9c006dae9abc admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:09:00.272132 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19280b5f-ec2e-463d-a346-9c006dae9abc admin admin] PATH_INFO: `/v3/projects/b3ab47ea6d88431d8d20fbbf818773b6` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:09:00.273316 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19280b5f-ec2e-463d-a346-9c006dae9abc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3ab47ea6d88431d8d20fbbf818773b6)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:09:00.275248 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19280b5f-ec2e-463d-a346-9c006dae9abc admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:09:00.322011 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-19280b5f-ec2e-463d-a346-9c006dae9abc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3ab47ea6d88431d8d20fbbf818773b6', 'request_id': 'req-19280b5f-ec2e-463d-a346-9c006dae9abc'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:09:00.329173 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-19280b5f-ec2e-463d-a346-9c006dae9abc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3ab47ea6d88431d8d20fbbf818773b6', 'request_id': 'req-19280b5f-ec2e-463d-a346-9c006dae9abc'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:09:00.334838 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-19280b5f-ec2e-463d-a346-9c006dae9abc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3ab47ea6d88431d8d20fbbf818773b6', 'request_id': 'req-19280b5f-ec2e-463d-a346-9c006dae9abc'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:09:00.346663 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-19280b5f-ec2e-463d-a346-9c006dae9abc admin admin] The token cache is being invalidate because project b3ab47ea6d88431d8d20fbbf818773b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:09:00.346950 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-19280b5f-ec2e-463d-a346-9c006dae9abc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:09:00.347615 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-19280b5f-ec2e-463d-a346-9c006dae9abc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:09:00.349148 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1141/2261] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:09:00 2022] DELETE /identity/v3/projects/b3ab47ea6d88431d8d20fbbf818773b6 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:09:06.651457 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13485d52-9749-46f0-8938-8fc2b1df167a service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:09:06.653079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-13485d52-9749-46f0-8938-8fc2b1df167a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:09:06.733887 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13485d52-9749-46f0-8938-8fc2b1df167a service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:09:06.794590 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:09:06.808124 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcedcc5-97b4-4fe2-9b09-f3bc8b0ae2c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:09:06.808777 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcedcc5-97b4-4fe2-9b09-f3bc8b0ae2c9 service nova] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:09:06.809060 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcedcc5-97b4-4fe2-9b09-f3bc8b0ae2c9 service nova] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:09:06.809345 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcedcc5-97b4-4fe2-9b09-f3bc8b0ae2c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:09:06.811768 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-9bcedcc5-97b4-4fe2-9b09-f3bc8b0ae2c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:09:06.823239 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcedcc5-97b4-4fe2-9b09-f3bc8b0ae2c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:09:06.824864 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcedcc5-97b4-4fe2-9b09-f3bc8b0ae2c9 service nova] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:09:06.835954 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1121/2262] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:09:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:09:08.920937 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0063a073-6591-4b42-ab32-b0e1cc6a05f5 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:09:08.921709 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0063a073-6591-4b42-ab32-b0e1cc6a05f5 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:09:08.921709 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0063a073-6591-4b42-ab32-b0e1cc6a05f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:09:08.940069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-0063a073-6591-4b42-ab32-b0e1cc6a05f5 None None] MFA Rules not processed for user `86619a80df8841a3ba7d8471ac38056a`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:09:08.941276 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-0063a073-6591-4b42-ab32-b0e1cc6a05f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:09:08.998127 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1142/2263] 10.222.0.13 () {58 vars in 1009 bytes} [Tue Jun 7 22:09:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:09:09.048972 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcedcc5-97b4-4fe2-9b09-f3bc8b0ae2c9 service nova] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:09:09.050432 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-9bcedcc5-97b4-4fe2-9b09-f3bc8b0ae2c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:09:09.134938 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcedcc5-97b4-4fe2-9b09-f3bc8b0ae2c9 service nova] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:09:09.140486 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabedd8b-a3dc-4b6e-89ac-e3dcdef9e6c0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1261563b6d4e50a4ffeea6a32bdb01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:09:09.141128 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabedd8b-a3dc-4b6e-89ac-e3dcdef9e6c0 service swift] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:09:09.141409 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabedd8b-a3dc-4b6e-89ac-e3dcdef9e6c0 service swift] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:09:09.141696 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabedd8b-a3dc-4b6e-89ac-e3dcdef9e6c0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:09:09.149749 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabedd8b-a3dc-4b6e-89ac-e3dcdef9e6c0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:09:09.151305 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabedd8b-a3dc-4b6e-89ac-e3dcdef9e6c0 service swift] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:09:09.160251 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1122/2264] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Jun 7 22:09:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 22:09:18.990454 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:09:19.157144 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=45 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:09:25.040214 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0063a073-6591-4b42-ab32-b0e1cc6a05f5 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:09:25.041620 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-0063a073-6591-4b42-ab32-b0e1cc6a05f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:09:25.105409 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0063a073-6591-4b42-ab32-b0e1cc6a05f5 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:09:25.109668 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27da21e-5670-43f7-8716-a0535412ec49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:09:25.110322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27da21e-5670-43f7-8716-a0535412ec49 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:09:25.110602 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27da21e-5670-43f7-8716-a0535412ec49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:09:25.110877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27da21e-5670-43f7-8716-a0535412ec49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:09:25.112689 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-b27da21e-5670-43f7-8716-a0535412ec49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:09:25.122564 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27da21e-5670-43f7-8716-a0535412ec49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:09:25.124027 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27da21e-5670-43f7-8716-a0535412ec49 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:09:25.132194 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1143/2265] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:09:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:09:25.242995 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312ef52a-1758-4e8b-9847-18c1d447d952 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:09:25.243740 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312ef52a-1758-4e8b-9847-18c1d447d952 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:09:25.243740 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312ef52a-1758-4e8b-9847-18c1d447d952 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:09:25.245188 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1123/2266] 10.222.0.13 () {54 vars in 921 bytes} [Tue Jun 7 22:09:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 22:09:25.252613 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1c386-5566-407a-83be-831bb901cb4a None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:09:25.252934 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1c386-5566-407a-83be-831bb901cb4a None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:09:25.253192 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1c386-5566-407a-83be-831bb901cb4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:09:25.293110 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-7ff1c386-5566-407a-83be-831bb901cb4a None None] MFA Rules not processed for user `dbe0303ea0764347bfb054870ebba41f`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:09:25.294971 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-7ff1c386-5566-407a-83be-831bb901cb4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:09:25.304500 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1144/2267] 10.222.0.13 () {58 vars in 1018 bytes} [Tue Jun 7 22:09:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:09:25.317122 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312ef52a-1758-4e8b-9847-18c1d447d952 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:09:25.318939 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-312ef52a-1758-4e8b-9847-18c1d447d952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:09:25.385274 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312ef52a-1758-4e8b-9847-18c1d447d952 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:09:25.389506 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef9bc6-e1ee-4baf-8e62-69a8a44ba518 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96987b04471944bc86ab44ff9936c732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:09:25.390219 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef9bc6-e1ee-4baf-8e62-69a8a44ba518 service glance] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:09:25.390542 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef9bc6-e1ee-4baf-8e62-69a8a44ba518 service glance] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:09:25.390823 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef9bc6-e1ee-4baf-8e62-69a8a44ba518 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:09:25.401086 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ef9bc6-e1ee-4baf-8e62-69a8a44ba518 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:09:25.403107 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ef9bc6-e1ee-4baf-8e62-69a8a44ba518 service glance] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:09:25.414229 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1124/2268] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:09:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:09:25.485680 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b5784-a886-409e-b4e7-ed63a681524b None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:09:25.485920 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b5784-a886-409e-b4e7-ed63a681524b None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:09:25.486200 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b5784-a886-409e-b4e7-ed63a681524b None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:09:25.487652 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1145/2269] 10.222.0.13 () {54 vars in 921 bytes} [Tue Jun 7 22:09:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 22:09:25.492541 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ba086-14c7-4e16-baa6-6f45680e1066 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:09:25.492878 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ba086-14c7-4e16-baa6-6f45680e1066 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:09:25.493189 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ba086-14c7-4e16-baa6-6f45680e1066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:09:25.511845 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-377ba086-14c7-4e16-baa6-6f45680e1066 None None] MFA Rules not processed for user `10242251e6284217bfb79c67b1567f75`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:09:25.513154 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-377ba086-14c7-4e16-baa6-6f45680e1066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:09:25.519970 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1125/2270] 10.222.0.13 () {58 vars in 1018 bytes} [Tue Jun 7 22:09:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:09:25.532605 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b5784-a886-409e-b4e7-ed63a681524b None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:09:25.534294 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-8c1b5784-a886-409e-b4e7-ed63a681524b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:09:25.547834 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b5784-a886-409e-b4e7-ed63a681524b None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:09:25.551278 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834144f7-39dd-4a39-9cd8-51709a531733 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:09:25.551936 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834144f7-39dd-4a39-9cd8-51709a531733 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:09:25.552216 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834144f7-39dd-4a39-9cd8-51709a531733 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:09:25.552485 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834144f7-39dd-4a39-9cd8-51709a531733 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:09:25.559556 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834144f7-39dd-4a39-9cd8-51709a531733 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:09:25.560998 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834144f7-39dd-4a39-9cd8-51709a531733 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:09:25.568522 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1146/2271] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:09:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:09:26.663807 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a06d66-1920-4274-a422-660ad776a8ca None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:09:26.664373 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a06d66-1920-4274-a422-660ad776a8ca None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:09:26.664479 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a06d66-1920-4274-a422-660ad776a8ca None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:09:26.665861 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1126/2272] 10.222.0.13 () {54 vars in 921 bytes} [Tue Jun 7 22:09:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 22:09:26.672007 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9093bbe-a883-4dde-ae79-a3998e7eaf38 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:09:26.672534 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9093bbe-a883-4dde-ae79-a3998e7eaf38 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:09:26.672634 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9093bbe-a883-4dde-ae79-a3998e7eaf38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:09:26.691635 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-d9093bbe-a883-4dde-ae79-a3998e7eaf38 None None] MFA Rules not processed for user `dbe0303ea0764347bfb054870ebba41f`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:09:26.692888 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d9093bbe-a883-4dde-ae79-a3998e7eaf38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:09:26.702842 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1147/2273] 10.222.0.13 () {58 vars in 1018 bytes} [Tue Jun 7 22:09:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:09:26.713135 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a06d66-1920-4274-a422-660ad776a8ca None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:09:26.725224 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a06d66-1920-4274-a422-660ad776a8ca None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:09:26.729240 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bafbe5-f806-40bd-a275-0216d9cecb16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:09:26.730038 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bafbe5-f806-40bd-a275-0216d9cecb16 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:09:26.730321 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bafbe5-f806-40bd-a275-0216d9cecb16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:09:26.731163 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bafbe5-f806-40bd-a275-0216d9cecb16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:09:26.742520 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bafbe5-f806-40bd-a275-0216d9cecb16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:09:26.744247 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bafbe5-f806-40bd-a275-0216d9cecb16 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:09:26.757235 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1127/2274] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:09:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:09:26.820834 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9093bbe-a883-4dde-ae79-a3998e7eaf38 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:09:26.822421 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d9093bbe-a883-4dde-ae79-a3998e7eaf38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:09:26.900498 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9093bbe-a883-4dde-ae79-a3998e7eaf38 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:09:26.906097 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cdcd49-6ce3-4756-9d05-8bbd1c51a648 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02941c6ad2d54e48b1114d9e030c66c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:09:26.906743 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cdcd49-6ce3-4756-9d05-8bbd1c51a648 service placement] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:09:26.907046 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cdcd49-6ce3-4756-9d05-8bbd1c51a648 service placement] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:09:26.907358 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cdcd49-6ce3-4756-9d05-8bbd1c51a648 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:09:26.909281 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-65cdcd49-6ce3-4756-9d05-8bbd1c51a648 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:09:26.922967 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cdcd49-6ce3-4756-9d05-8bbd1c51a648 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:09:26.924851 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cdcd49-6ce3-4756-9d05-8bbd1c51a648 service placement] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:09:26.936341 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1148/2275] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:09:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:09:32.739034 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8844a6-ba05-4f36-8d11-7a2e13a3dad8 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:09:32.739540 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8844a6-ba05-4f36-8d11-7a2e13a3dad8 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:09:32.739642 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8844a6-ba05-4f36-8d11-7a2e13a3dad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:09:32.757565 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-3c8844a6-ba05-4f36-8d11-7a2e13a3dad8 None None] MFA Rules not processed for user `86619a80df8841a3ba7d8471ac38056a`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:09:32.758769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-3c8844a6-ba05-4f36-8d11-7a2e13a3dad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:09:32.766126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1128/2276] 10.222.0.13 () {58 vars in 1009 bytes} [Tue Jun 7 22:09:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:09:32.777617 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cdcd49-6ce3-4756-9d05-8bbd1c51a648 service placement] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:09:32.788891 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cdcd49-6ce3-4756-9d05-8bbd1c51a648 service placement] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:09:32.794343 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0851a1-bc50-4fc2-a93f-55e72986a085 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1261563b6d4e50a4ffeea6a32bdb01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:09:32.795004 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0851a1-bc50-4fc2-a93f-55e72986a085 service swift] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:09:32.795294 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0851a1-bc50-4fc2-a93f-55e72986a085 service swift] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:09:32.795639 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0851a1-bc50-4fc2-a93f-55e72986a085 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:09:32.803621 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0851a1-bc50-4fc2-a93f-55e72986a085 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:09:32.805193 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0851a1-bc50-4fc2-a93f-55e72986a085 service swift] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:09:32.813804 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1149/2277] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Jun 7 22:09:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 22:09:39.740516 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8844a6-ba05-4f36-8d11-7a2e13a3dad8 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:09:39.742011 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-3c8844a6-ba05-4f36-8d11-7a2e13a3dad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:09:39.772398 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8844a6-ba05-4f36-8d11-7a2e13a3dad8 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:09:39.778823 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd6c705-a829-4547-bddb-9dc07cf670f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:09:39.779519 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6c705-a829-4547-bddb-9dc07cf670f4 service nova] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:09:39.779803 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6c705-a829-4547-bddb-9dc07cf670f4 service nova] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:09:39.780079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6c705-a829-4547-bddb-9dc07cf670f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:09:39.781929 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-6dd6c705-a829-4547-bddb-9dc07cf670f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:09:39.843810 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd6c705-a829-4547-bddb-9dc07cf670f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:09:39.845397 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd6c705-a829-4547-bddb-9dc07cf670f4 service nova] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:09:39.854930 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1129/2278] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:09:39 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:09:42.810648 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=82 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:09:49.850164 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=66 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:09:59.710599 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4444d-c43d-4a4e-97db-a1bbd9cafdd4 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:09:59.711526 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4444d-c43d-4a4e-97db-a1bbd9cafdd4 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:09:59.711822 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4444d-c43d-4a4e-97db-a1bbd9cafdd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:09:59.731670 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-62d4444d-c43d-4a4e-97db-a1bbd9cafdd4 None None] MFA Rules not processed for user `86619a80df8841a3ba7d8471ac38056a`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:09:59.732926 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-62d4444d-c43d-4a4e-97db-a1bbd9cafdd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:09:59.740115 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1150/2279] 10.222.0.13 () {58 vars in 967 bytes} [Tue Jun 7 22:09:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:09:59.753659 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd6c705-a829-4547-bddb-9dc07cf670f4 service nova] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:09:59.765425 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd6c705-a829-4547-bddb-9dc07cf670f4 service nova] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:09:59.772239 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3093d983-13a2-4233-b728-a387efbe5d97 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1261563b6d4e50a4ffeea6a32bdb01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:09:59.772907 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093d983-13a2-4233-b728-a387efbe5d97 service swift] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:09:59.773199 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093d983-13a2-4233-b728-a387efbe5d97 service swift] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:09:59.773482 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093d983-13a2-4233-b728-a387efbe5d97 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:09:59.782668 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3093d983-13a2-4233-b728-a387efbe5d97 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:09:59.784289 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3093d983-13a2-4233-b728-a387efbe5d97 service swift] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:09:59.792439 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1130/2280] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Jun 7 22:09:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 22:10:07.607415 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d4444d-c43d-4a4e-97db-a1bbd9cafdd4 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:10:07.608752 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-62d4444d-c43d-4a4e-97db-a1bbd9cafdd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:10:07.624038 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d4444d-c43d-4a4e-97db-a1bbd9cafdd4 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:10:07.629439 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65763a29-a65e-4a2c-b469-fe89555c9559 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:10:07.630072 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65763a29-a65e-4a2c-b469-fe89555c9559 service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:10:07.630361 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65763a29-a65e-4a2c-b469-fe89555c9559 service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:10:07.630637 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65763a29-a65e-4a2c-b469-fe89555c9559 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:10:07.632356 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-65763a29-a65e-4a2c-b469-fe89555c9559 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:10:07.690216 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65763a29-a65e-4a2c-b469-fe89555c9559 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:10:07.691862 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65763a29-a65e-4a2c-b469-fe89555c9559 service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:10:07.701082 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1151/2281] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:10:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:10:09.789132 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=6 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:10:11.161556 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3093d983-13a2-4233-b728-a387efbe5d97 service swift] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:10:11.176969 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3093d983-13a2-4233-b728-a387efbe5d97 service swift] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:10:11.184304 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e256c-f0e1-4460-8859-9edc4fb88f9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:10:11.184731 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e256c-f0e1-4460-8859-9edc4fb88f9b service nova] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:10:11.184998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e256c-f0e1-4460-8859-9edc4fb88f9b service nova] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:10:11.185468 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e256c-f0e1-4460-8859-9edc4fb88f9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:10:11.187458 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-7e5e256c-f0e1-4460-8859-9edc4fb88f9b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:10:11.204198 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5e256c-f0e1-4460-8859-9edc4fb88f9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:10:11.205795 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5e256c-f0e1-4460-8859-9edc4fb88f9b service nova] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:10:11.217040 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1131/2282] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:10:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:10:16.769953 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c11837-0d71-4521-a1e7-bd5522ca5c94 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:10:16.770509 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c11837-0d71-4521-a1e7-bd5522ca5c94 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:10:16.770604 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c11837-0d71-4521-a1e7-bd5522ca5c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:10:16.789186 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-b7c11837-0d71-4521-a1e7-bd5522ca5c94 None None] MFA Rules not processed for user `c44c1788959e4b339c9cd5aad4318496`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:10:16.790527 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-b7c11837-0d71-4521-a1e7-bd5522ca5c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:10:16.798573 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1152/2283] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:10:16 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:10:21.212323 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=6 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:10:25.323570 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210dc571-f55b-4155-b555-43defa122c15 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:10:25.324189 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210dc571-f55b-4155-b555-43defa122c15 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:10:25.324189 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210dc571-f55b-4155-b555-43defa122c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:10:25.344423 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-210dc571-f55b-4155-b555-43defa122c15 None None] MFA Rules not processed for user `607b70b4d6e14c95ae2880dd3577c8ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:10:25.345653 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-210dc571-f55b-4155-b555-43defa122c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:10:25.352969 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1132/2284] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:10:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:10:25.661432 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c11837-0d71-4521-a1e7-bd5522ca5c94 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:10:25.666063 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=35 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:10:25.672875 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c11837-0d71-4521-a1e7-bd5522ca5c94 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:10:25.676368 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ad3d26-eef3-4ace-9be3-cf2bf011b8e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:10:25.677001 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ad3d26-eef3-4ace-9be3-cf2bf011b8e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:10:25.677314 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ad3d26-eef3-4ace-9be3-cf2bf011b8e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:10:25.677550 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ad3d26-eef3-4ace-9be3-cf2bf011b8e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:10:25.679284 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-75ad3d26-eef3-4ace-9be3-cf2bf011b8e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:10:25.689229 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ad3d26-eef3-4ace-9be3-cf2bf011b8e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:10:25.690660 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ad3d26-eef3-4ace-9be3-cf2bf011b8e9 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:10:25.698537 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1153/2285] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:10:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:10:35.338042 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:10:35.695221 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=5 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:10:37.006642 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeafb43-9721-43cc-bf4d-9098d8e7e9a2 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:10:37.007091 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeafb43-9721-43cc-bf4d-9098d8e7e9a2 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:10:37.007196 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeafb43-9721-43cc-bf4d-9098d8e7e9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:10:37.025006 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-7eeafb43-9721-43cc-bf4d-9098d8e7e9a2 None None] MFA Rules not processed for user `c710942802df49ec8bd630e4b225eed2`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:10:37.026127 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-7eeafb43-9721-43cc-bf4d-9098d8e7e9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:10:37.081236 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1133/2286] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:10:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:10:47.074074 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=25 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:10:53.069643 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ad3d26-eef3-4ace-9be3-cf2bf011b8e9 service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:10:53.083721 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ad3d26-eef3-4ace-9be3-cf2bf011b8e9 service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:10:53.089217 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d2fa90-ec1e-4206-aa5f-ab2ab091e277 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:10:53.089858 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2fa90-ec1e-4206-aa5f-ab2ab091e277 service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:10:53.090144 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2fa90-ec1e-4206-aa5f-ab2ab091e277 service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:10:53.090422 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2fa90-ec1e-4206-aa5f-ab2ab091e277 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:10:53.092933 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-53d2fa90-ec1e-4206-aa5f-ab2ab091e277 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:10:53.103845 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d2fa90-ec1e-4206-aa5f-ab2ab091e277 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:10:53.105426 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d2fa90-ec1e-4206-aa5f-ab2ab091e277 service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:10:53.115106 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1154/2287] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:10:53 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:11:03.110979 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=6 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:11:07.841605 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeafb43-9721-43cc-bf4d-9098d8e7e9a2 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:11:07.855297 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeafb43-9721-43cc-bf4d-9098d8e7e9a2 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:11:07.860816 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38de5d6d-54e4-47f1-8fdb-6637aee480e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:11:07.861451 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de5d6d-54e4-47f1-8fdb-6637aee480e7 service nova] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:07.861731 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de5d6d-54e4-47f1-8fdb-6637aee480e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:07.862006 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de5d6d-54e4-47f1-8fdb-6637aee480e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:07.863784 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-38de5d6d-54e4-47f1-8fdb-6637aee480e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:11:07.923047 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38de5d6d-54e4-47f1-8fdb-6637aee480e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:11:07.924685 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38de5d6d-54e4-47f1-8fdb-6637aee480e7 service nova] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:11:07.933636 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1134/2288] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:11:07 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:11:09.571140 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d2fa90-ec1e-4206-aa5f-ab2ab091e277 service nova] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:11:09.582870 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d2fa90-ec1e-4206-aa5f-ab2ab091e277 service nova] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:11:09.589079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b527017-c00a-4672-9729-17340b235846 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:11:09.589712 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b527017-c00a-4672-9729-17340b235846 service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:09.589995 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b527017-c00a-4672-9729-17340b235846 service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:09.590272 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b527017-c00a-4672-9729-17340b235846 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:09.592158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-5b527017-c00a-4672-9729-17340b235846 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:11:09.650236 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b527017-c00a-4672-9729-17340b235846 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:11:09.651852 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b527017-c00a-4672-9729-17340b235846 service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:11:09.660826 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1155/2289] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:11:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:11:11.457080 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38de5d6d-54e4-47f1-8fdb-6637aee480e7 service nova] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:11:11.470508 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38de5d6d-54e4-47f1-8fdb-6637aee480e7 service nova] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:11:11.476337 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f78360a-849b-492f-9b0b-32c4cb0cb89f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:11:11.477465 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f78360a-849b-492f-9b0b-32c4cb0cb89f service nova] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:11.477921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f78360a-849b-492f-9b0b-32c4cb0cb89f service nova] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:11.478196 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f78360a-849b-492f-9b0b-32c4cb0cb89f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:11.480231 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-2f78360a-849b-492f-9b0b-32c4cb0cb89f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:11:11.494569 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f78360a-849b-492f-9b0b-32c4cb0cb89f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:11:11.496198 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f78360a-849b-492f-9b0b-32c4cb0cb89f service nova] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:11:11.508932 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1135/2290] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:11:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 22:11:18.489565 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443eb1c-80b1-4270-b138-c87ceb11d5fc None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:18.490163 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443eb1c-80b1-4270-b138-c87ceb11d5fc None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:18.490163 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443eb1c-80b1-4270-b138-c87ceb11d5fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:18.507980 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-d443eb1c-80b1-4270-b138-c87ceb11d5fc None None] MFA Rules not processed for user `de021dc74a4446d8b6a63883fd56a7c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:11:18.509182 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d443eb1c-80b1-4270-b138-c87ceb11d5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:11:18.516519 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1156/2291] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:11:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:11:20.950064 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877e759-c226-4302-8608-a5f240110fc9 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:20.950545 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877e759-c226-4302-8608-a5f240110fc9 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:20.950766 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877e759-c226-4302-8608-a5f240110fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:20.967916 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-6877e759-c226-4302-8608-a5f240110fc9 None None] MFA Rules not processed for user `86619a80df8841a3ba7d8471ac38056a`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:11:20.969762 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-6877e759-c226-4302-8608-a5f240110fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:11:20.978627 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1136/2292] 10.222.0.13 () {58 vars in 967 bytes} [Tue Jun 7 22:11:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:11:21.030805 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d443eb1c-80b1-4270-b138-c87ceb11d5fc None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:11:21.048196 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d443eb1c-80b1-4270-b138-c87ceb11d5fc None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:11:21.053499 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ee968-17db-4128-9504-f2c60d522149 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1261563b6d4e50a4ffeea6a32bdb01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:11:21.054148 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ee968-17db-4128-9504-f2c60d522149 service swift] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:21.054427 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ee968-17db-4128-9504-f2c60d522149 service swift] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:21.054696 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ee968-17db-4128-9504-f2c60d522149 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:21.063753 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2ee968-17db-4128-9504-f2c60d522149 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:11:21.065334 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2ee968-17db-4128-9504-f2c60d522149 service swift] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:11:21.074281 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1157/2293] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Jun 7 22:11:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 22:11:21.445935 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877e759-c226-4302-8608-a5f240110fc9 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:11:21.456628 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877e759-c226-4302-8608-a5f240110fc9 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:11:21.461790 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85783ccc-5324-4974-adf0-bee0803e528a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:11:21.462391 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85783ccc-5324-4974-adf0-bee0803e528a service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:21.462696 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85783ccc-5324-4974-adf0-bee0803e528a service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:21.462961 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85783ccc-5324-4974-adf0-bee0803e528a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:21.465057 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-85783ccc-5324-4974-adf0-bee0803e528a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:11:21.560029 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85783ccc-5324-4974-adf0-bee0803e528a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:11:21.562115 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85783ccc-5324-4974-adf0-bee0803e528a service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:11:21.574947 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1137/2294] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:11:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:11:25.029128 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ee968-17db-4128-9504-f2c60d522149 service swift] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:11:25.030811 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-2d2ee968-17db-4128-9504-f2c60d522149 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:11:25.060873 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ee968-17db-4128-9504-f2c60d522149 service swift] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:11:25.066631 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2528fb7-811c-4db4-8222-45b8aab9409a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02941c6ad2d54e48b1114d9e030c66c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:11:25.067477 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2528fb7-811c-4db4-8222-45b8aab9409a service placement] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:25.068020 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2528fb7-811c-4db4-8222-45b8aab9409a service placement] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:25.068391 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2528fb7-811c-4db4-8222-45b8aab9409a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:25.072836 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d2528fb7-811c-4db4-8222-45b8aab9409a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:11:25.089416 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2528fb7-811c-4db4-8222-45b8aab9409a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:11:25.095021 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2528fb7-811c-4db4-8222-45b8aab9409a service placement] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:11:25.112567 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1158/2295] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:11:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:11:28.855642 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85783ccc-5324-4974-adf0-bee0803e528a service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:11:28.857490 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-85783ccc-5324-4974-adf0-bee0803e528a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:11:28.872112 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85783ccc-5324-4974-adf0-bee0803e528a service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:11:28.877642 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29615141-3899-4b8e-9cbd-4036c54068f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:11:28.878545 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29615141-3899-4b8e-9cbd-4036c54068f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:28.878851 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29615141-3899-4b8e-9cbd-4036c54068f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:28.879131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29615141-3899-4b8e-9cbd-4036c54068f8 admin admin] PATH_INFO: `/v3/users/7af02cd6ffd0484c8f6569629cd6313c` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:28.880394 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29615141-3899-4b8e-9cbd-4036c54068f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7af02cd6ffd0484c8f6569629cd6313c)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:11:28.882449 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29615141-3899-4b8e-9cbd-4036c54068f8 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:11:28.936800 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-29615141-3899-4b8e-9cbd-4036c54068f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7af02cd6ffd0484c8f6569629cd6313c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:11:28.938165 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-29615141-3899-4b8e-9cbd-4036c54068f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7af02cd6ffd0484c8f6569629cd6313c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:11:28.945612 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-29615141-3899-4b8e-9cbd-4036c54068f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7af02cd6ffd0484c8f6569629cd6313c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:11:28.956008 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:11:28.958557 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-29615141-3899-4b8e-9cbd-4036c54068f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7af02cd6ffd0484c8f6569629cd6313c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:11:28.967264 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-29615141-3899-4b8e-9cbd-4036c54068f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7af02cd6ffd0484c8f6569629cd6313c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:11:28.974067 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1138/2296] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:11:28 2022] DELETE /identity/v3/users/7af02cd6ffd0484c8f6569629cd6313c => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:11:29.459382 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2528fb7-811c-4db4-8222-45b8aab9409a service placement] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:11:29.472948 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2528fb7-811c-4db4-8222-45b8aab9409a service placement] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:11:29.531825 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092511bd-f0ab-4c15-9c82-027e16d16616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:11:29.532831 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092511bd-f0ab-4c15-9c82-027e16d16616 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:29.533115 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092511bd-f0ab-4c15-9c82-027e16d16616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:29.533393 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092511bd-f0ab-4c15-9c82-027e16d16616 admin admin] PATH_INFO: `/v3/projects/bd31cd38d46643fbac516d21ee4f89ac` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:29.534569 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092511bd-f0ab-4c15-9c82-027e16d16616 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd31cd38d46643fbac516d21ee4f89ac)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:11:29.536504 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092511bd-f0ab-4c15-9c82-027e16d16616 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:11:29.581007 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-092511bd-f0ab-4c15-9c82-027e16d16616 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd31cd38d46643fbac516d21ee4f89ac', 'request_id': 'req-092511bd-f0ab-4c15-9c82-027e16d16616'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:11:29.589191 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-092511bd-f0ab-4c15-9c82-027e16d16616 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd31cd38d46643fbac516d21ee4f89ac', 'request_id': 'req-092511bd-f0ab-4c15-9c82-027e16d16616'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:11:29.595600 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-092511bd-f0ab-4c15-9c82-027e16d16616 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd31cd38d46643fbac516d21ee4f89ac', 'request_id': 'req-092511bd-f0ab-4c15-9c82-027e16d16616'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:11:29.597649 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:11:29.610485 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-092511bd-f0ab-4c15-9c82-027e16d16616 admin admin] The token cache is being invalidate because project bd31cd38d46643fbac516d21ee4f89ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:11:29.610838 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-092511bd-f0ab-4c15-9c82-027e16d16616 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:11:29.611693 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-092511bd-f0ab-4c15-9c82-027e16d16616 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:11:29.613267 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1159/2297] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:11:29 2022] DELETE /identity/v3/projects/bd31cd38d46643fbac516d21ee4f89ac => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:11:29.617180 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29615141-3899-4b8e-9cbd-4036c54068f8 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:11:29.618570 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-29615141-3899-4b8e-9cbd-4036c54068f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:11:29.631764 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29615141-3899-4b8e-9cbd-4036c54068f8 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:11:29.690443 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa04d9-bd35-4d63-ae40-516a7367d426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:11:29.691313 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa04d9-bd35-4d63-ae40-516a7367d426 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:29.691626 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa04d9-bd35-4d63-ae40-516a7367d426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:29.691906 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa04d9-bd35-4d63-ae40-516a7367d426 admin admin] PATH_INFO: `/v3/users/0388de1feaf74c9b8073f04b276dc310` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:29.693120 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fa04d9-bd35-4d63-ae40-516a7367d426 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0388de1feaf74c9b8073f04b276dc310)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:11:29.695071 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fa04d9-bd35-4d63-ae40-516a7367d426 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:11:29.741225 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-36fa04d9-bd35-4d63-ae40-516a7367d426 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0388de1feaf74c9b8073f04b276dc310'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:11:29.741879 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-36fa04d9-bd35-4d63-ae40-516a7367d426 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0388de1feaf74c9b8073f04b276dc310'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:11:29.753082 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-36fa04d9-bd35-4d63-ae40-516a7367d426 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0388de1feaf74c9b8073f04b276dc310'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:11:29.766053 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-36fa04d9-bd35-4d63-ae40-516a7367d426 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0388de1feaf74c9b8073f04b276dc310'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:11:29.772871 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-36fa04d9-bd35-4d63-ae40-516a7367d426 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0388de1feaf74c9b8073f04b276dc310'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:11:29.778801 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1139/2298] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:11:29 2022] DELETE /identity/v3/users/0388de1feaf74c9b8073f04b276dc310 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:11:29.953969 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092511bd-f0ab-4c15-9c82-027e16d16616 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:11:29.955526 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-092511bd-f0ab-4c15-9c82-027e16d16616 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:11:29.968880 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092511bd-f0ab-4c15-9c82-027e16d16616 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:11:30.032646 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b92cb1-2325-4ce3-a8b9-5dd3cc35db9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:11:30.033511 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b92cb1-2325-4ce3-a8b9-5dd3cc35db9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:30.033815 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b92cb1-2325-4ce3-a8b9-5dd3cc35db9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:30.034087 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b92cb1-2325-4ce3-a8b9-5dd3cc35db9a admin admin] PATH_INFO: `/v3/projects/7ba15b864eac4c0081b6edc6f8f45105` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:30.035261 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b92cb1-2325-4ce3-a8b9-5dd3cc35db9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ba15b864eac4c0081b6edc6f8f45105)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:11:30.037225 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b92cb1-2325-4ce3-a8b9-5dd3cc35db9a admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:11:30.083025 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-60b92cb1-2325-4ce3-a8b9-5dd3cc35db9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ba15b864eac4c0081b6edc6f8f45105', 'request_id': 'req-60b92cb1-2325-4ce3-a8b9-5dd3cc35db9a'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:11:30.090368 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-60b92cb1-2325-4ce3-a8b9-5dd3cc35db9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ba15b864eac4c0081b6edc6f8f45105', 'request_id': 'req-60b92cb1-2325-4ce3-a8b9-5dd3cc35db9a'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:11:30.096158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-60b92cb1-2325-4ce3-a8b9-5dd3cc35db9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ba15b864eac4c0081b6edc6f8f45105', 'request_id': 'req-60b92cb1-2325-4ce3-a8b9-5dd3cc35db9a'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:11:30.107206 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-60b92cb1-2325-4ce3-a8b9-5dd3cc35db9a admin admin] The token cache is being invalidate because project 7ba15b864eac4c0081b6edc6f8f45105 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:11:30.107522 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-60b92cb1-2325-4ce3-a8b9-5dd3cc35db9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:11:30.108175 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-60b92cb1-2325-4ce3-a8b9-5dd3cc35db9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:11:30.110438 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1160/2299] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:11:29 2022] DELETE /identity/v3/projects/7ba15b864eac4c0081b6edc6f8f45105 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:11:30.115522 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c5680-67a4-42eb-a818-d994f0bad962 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:30.115975 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c5680-67a4-42eb-a818-d994f0bad962 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:30.116067 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c5680-67a4-42eb-a818-d994f0bad962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:30.134075 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-486c5680-67a4-42eb-a818-d994f0bad962 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:11:30.135565 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-486c5680-67a4-42eb-a818-d994f0bad962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:11:30.193828 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1140/2300] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:11:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:11:30.201338 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aacb70a-36e7-47d5-a7a1-0807aa793bdc None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:30.201648 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aacb70a-36e7-47d5-a7a1-0807aa793bdc None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:30.201986 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aacb70a-36e7-47d5-a7a1-0807aa793bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:30.221035 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-6aacb70a-36e7-47d5-a7a1-0807aa793bdc None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:11:30.222240 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-6aacb70a-36e7-47d5-a7a1-0807aa793bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:11:30.230264 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1161/2301] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:11:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:11:30.234325 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c5680-67a4-42eb-a818-d994f0bad962 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:11:30.240482 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:11:30.245876 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c5680-67a4-42eb-a818-d994f0bad962 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:11:30.255500 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e0593d-fb23-4857-af70-73a1cbda8c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:11:30.257019 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e0593d-fb23-4857-af70-73a1cbda8c44 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:30.257647 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e0593d-fb23-4857-af70-73a1cbda8c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:30.258191 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e0593d-fb23-4857-af70-73a1cbda8c44 admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:30.259299 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e0593d-fb23-4857-af70-73a1cbda8c44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:11:30.259821 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e0593d-fb23-4857-af70-73a1cbda8c44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:11:30.261553 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e0593d-fb23-4857-af70-73a1cbda8c44 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:11:30.274118 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1141/2302] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:11:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:11:30.278634 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aacb70a-36e7-47d5-a7a1-0807aa793bdc None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:11:30.288807 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aacb70a-36e7-47d5-a7a1-0807aa793bdc None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:11:30.293498 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60846044-e4a6-4ce3-a1ba-1e0b9c7f72ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:11:30.294149 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60846044-e4a6-4ce3-a1ba-1e0b9c7f72ff admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:30.294431 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60846044-e4a6-4ce3-a1ba-1e0b9c7f72ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:30.294706 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60846044-e4a6-4ce3-a1ba-1e0b9c7f72ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:30.295447 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60846044-e4a6-4ce3-a1ba-1e0b9c7f72ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:11:30.297651 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60846044-e4a6-4ce3-a1ba-1e0b9c7f72ff admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:11:30.332362 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1162/2303] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:11:30 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:11:30.336536 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e0593d-fb23-4857-af70-73a1cbda8c44 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:11:30.347000 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e0593d-fb23-4857-af70-73a1cbda8c44 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:11:30.403284 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d136eb7-3dd9-4f9d-a82e-19e9057b5b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:11:30.404000 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d136eb7-3dd9-4f9d-a82e-19e9057b5b87 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:30.404295 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d136eb7-3dd9-4f9d-a82e-19e9057b5b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:30.404601 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d136eb7-3dd9-4f9d-a82e-19e9057b5b87 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:30.405302 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d136eb7-3dd9-4f9d-a82e-19e9057b5b87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:11:30.407914 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d136eb7-3dd9-4f9d-a82e-19e9057b5b87 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:11:30.427216 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1142/2304] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:11:30 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:11:30.431380 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60846044-e4a6-4ce3-a1ba-1e0b9c7f72ff admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:11:30.442116 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60846044-e4a6-4ce3-a1ba-1e0b9c7f72ff admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:11:30.446659 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f313f-5bc5-4d34-96ad-d263b5280ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:11:30.447314 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f313f-5bc5-4d34-96ad-d263b5280ba8 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:30.447627 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f313f-5bc5-4d34-96ad-d263b5280ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:30.447905 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f313f-5bc5-4d34-96ad-d263b5280ba8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:30.448627 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918f313f-5bc5-4d34-96ad-d263b5280ba8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:11:30.450237 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918f313f-5bc5-4d34-96ad-d263b5280ba8 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:11:30.461316 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1163/2305] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:11:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:11:30.465252 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d136eb7-3dd9-4f9d-a82e-19e9057b5b87 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:11:30.476074 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d136eb7-3dd9-4f9d-a82e-19e9057b5b87 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:11:30.481402 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e227fb3-8a71-4de3-aa96-b5eca2531e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:11:30.482089 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e227fb3-8a71-4de3-aa96-b5eca2531e82 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:30.482362 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e227fb3-8a71-4de3-aa96-b5eca2531e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:30.482643 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e227fb3-8a71-4de3-aa96-b5eca2531e82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:30.483469 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e227fb3-8a71-4de3-aa96-b5eca2531e82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:11:30.485233 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e227fb3-8a71-4de3-aa96-b5eca2531e82 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:11:30.495290 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1143/2306] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:11:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:11:30.500295 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f313f-5bc5-4d34-96ad-d263b5280ba8 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:11:30.510555 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f313f-5bc5-4d34-96ad-d263b5280ba8 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:11:30.516605 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669da2c9-5a9b-4c8e-8f06-3d12737c8815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:11:30.517105 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669da2c9-5a9b-4c8e-8f06-3d12737c8815 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:30.517453 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669da2c9-5a9b-4c8e-8f06-3d12737c8815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:30.517795 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669da2c9-5a9b-4c8e-8f06-3d12737c8815 admin admin] PATH_INFO: `/v3/projects/98914183afee4e58ad7a425cb019e639/users/7e8c46b1820a4ce5ac853ab1cc88fc30/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:30.533543 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669da2c9-5a9b-4c8e-8f06-3d12737c8815 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98914183afee4e58ad7a425cb019e639, user_id=7e8c46b1820a4ce5ac853ab1cc88fc30, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:11:30.536281 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669da2c9-5a9b-4c8e-8f06-3d12737c8815 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:11:30.550355 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1164/2307] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:11:30 2022] PUT /identity/v3/projects/98914183afee4e58ad7a425cb019e639/users/7e8c46b1820a4ce5ac853ab1cc88fc30/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:11:30.555158 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4793a3f-9f33-443c-8a27-277b576a2610 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:30.555481 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4793a3f-9f33-443c-8a27-277b576a2610 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:30.555761 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4793a3f-9f33-443c-8a27-277b576a2610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:30.574299 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-d4793a3f-9f33-443c-8a27-277b576a2610 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:11:30.575548 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-d4793a3f-9f33-443c-8a27-277b576a2610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:11:30.647586 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1144/2308] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:11:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:11:30.657575 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669da2c9-5a9b-4c8e-8f06-3d12737c8815 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:11:30.659292 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-669da2c9-5a9b-4c8e-8f06-3d12737c8815 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:11:30.674572 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:11:30.720946 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669da2c9-5a9b-4c8e-8f06-3d12737c8815 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:11:30.724619 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e858cc8f-be48-46fa-a1cf-ebe59918df94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:11:30.725263 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858cc8f-be48-46fa-a1cf-ebe59918df94 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:30.725544 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858cc8f-be48-46fa-a1cf-ebe59918df94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:30.725816 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858cc8f-be48-46fa-a1cf-ebe59918df94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:30.736430 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e858cc8f-be48-46fa-a1cf-ebe59918df94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:11:30.737860 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e858cc8f-be48-46fa-a1cf-ebe59918df94 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:11:30.747716 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1165/2309] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:11:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 22:11:35.834106 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354b83f-fef8-48f7-9d4c-342a29e7bbda None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:35.834578 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354b83f-fef8-48f7-9d4c-342a29e7bbda None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:35.834694 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354b83f-fef8-48f7-9d4c-342a29e7bbda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:35.863587 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-e354b83f-fef8-48f7-9d4c-342a29e7bbda None None] MFA Rules not processed for user `7e8c46b1820a4ce5ac853ab1cc88fc30`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:11:35.864716 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-e354b83f-fef8-48f7-9d4c-342a29e7bbda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:11:35.944022 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:11:35.955796 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e858cc8f-be48-46fa-a1cf-ebe59918df94 service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:11:35.957215 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1145/2310] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:11:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:11:35.967691 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e858cc8f-be48-46fa-a1cf-ebe59918df94 service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:11:35.972888 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e9fb4e-b40f-40ca-9eea-05e89bdb8b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:11:35.973553 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9fb4e-b40f-40ca-9eea-05e89bdb8b68 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:35.973841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9fb4e-b40f-40ca-9eea-05e89bdb8b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:35.974120 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9fb4e-b40f-40ca-9eea-05e89bdb8b68 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:35.974855 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e9fb4e-b40f-40ca-9eea-05e89bdb8b68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:11:35.977075 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e9fb4e-b40f-40ca-9eea-05e89bdb8b68 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:11:36.011511 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1166/2311] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:11:35 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:11:36.016109 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e354b83f-fef8-48f7-9d4c-342a29e7bbda None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:11:36.028700 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e354b83f-fef8-48f7-9d4c-342a29e7bbda None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:11:36.097206 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a963c5-4817-4685-97c9-54d85c7a90b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:11:36.097890 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a963c5-4817-4685-97c9-54d85c7a90b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:36.098174 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a963c5-4817-4685-97c9-54d85c7a90b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:36.098470 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a963c5-4817-4685-97c9-54d85c7a90b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:36.099151 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a963c5-4817-4685-97c9-54d85c7a90b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:11:36.101152 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a963c5-4817-4685-97c9-54d85c7a90b7 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:11:36.128113 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1146/2312] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:11:36 2022] POST /identity/v3/users => generated 292 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:11:36.132235 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e9fb4e-b40f-40ca-9eea-05e89bdb8b68 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:11:36.143797 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e9fb4e-b40f-40ca-9eea-05e89bdb8b68 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:11:36.148790 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72703022-3fe6-45dd-b034-b465fcaf595f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:11:36.149463 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72703022-3fe6-45dd-b034-b465fcaf595f admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:36.149750 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72703022-3fe6-45dd-b034-b465fcaf595f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:36.150027 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72703022-3fe6-45dd-b034-b465fcaf595f admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:36.150756 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72703022-3fe6-45dd-b034-b465fcaf595f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:11:36.152414 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72703022-3fe6-45dd-b034-b465fcaf595f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:11:36.164991 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1167/2313] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:11:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:11:36.175047 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a963c5-4817-4685-97c9-54d85c7a90b7 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:11:36.188100 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a963c5-4817-4685-97c9-54d85c7a90b7 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:11:36.208231 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa34a53-e880-40a9-82d8-d24a7348727e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:11:36.209222 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa34a53-e880-40a9-82d8-d24a7348727e admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:36.209430 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa34a53-e880-40a9-82d8-d24a7348727e admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:36.210549 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa34a53-e880-40a9-82d8-d24a7348727e admin admin] PATH_INFO: `/v3/projects/483ce8210232462f81b34fe3973a3cfd/users/df8302231ba9470c99b27bd0048e983b/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:36.228649 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa34a53-e880-40a9-82d8-d24a7348727e admin admin] RBAC: Authorizing `identity:create_grant(project_id=483ce8210232462f81b34fe3973a3cfd, user_id=df8302231ba9470c99b27bd0048e983b, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:11:36.231267 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa34a53-e880-40a9-82d8-d24a7348727e admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:11:36.245223 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1147/2314] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:11:36 2022] PUT /identity/v3/projects/483ce8210232462f81b34fe3973a3cfd/users/df8302231ba9470c99b27bd0048e983b/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:11:36.249103 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72703022-3fe6-45dd-b034-b465fcaf595f admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:11:36.272825 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72703022-3fe6-45dd-b034-b465fcaf595f admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:11:36.355854 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef65c2d3-14d6-4574-9229-e3a8990f5f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:11:36.356636 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65c2d3-14d6-4574-9229-e3a8990f5f7c admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:36.356991 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65c2d3-14d6-4574-9229-e3a8990f5f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:36.357277 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65c2d3-14d6-4574-9229-e3a8990f5f7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:36.358083 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef65c2d3-14d6-4574-9229-e3a8990f5f7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:11:36.360711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef65c2d3-14d6-4574-9229-e3a8990f5f7c admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:11:36.374437 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1168/2315] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:11:36 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:11:36.379180 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa34a53-e880-40a9-82d8-d24a7348727e admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:11:36.398157 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa34a53-e880-40a9-82d8-d24a7348727e admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:11:36.409140 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ca00d-5f8d-4385-aab4-1f091e24dc2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:11:36.410315 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ca00d-5f8d-4385-aab4-1f091e24dc2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:36.410772 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ca00d-5f8d-4385-aab4-1f091e24dc2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:36.411838 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ca00d-5f8d-4385-aab4-1f091e24dc2a admin admin] PATH_INFO: `/v3/projects/483ce8210232462f81b34fe3973a3cfd/users/df8302231ba9470c99b27bd0048e983b/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:36.414338 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ca00d-5f8d-4385-aab4-1f091e24dc2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=483ce8210232462f81b34fe3973a3cfd, user_id=df8302231ba9470c99b27bd0048e983b, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:11:36.417199 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ca00d-5f8d-4385-aab4-1f091e24dc2a admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:11:36.444981 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1148/2316] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:11:36 2022] PUT /identity/v3/projects/483ce8210232462f81b34fe3973a3cfd/users/df8302231ba9470c99b27bd0048e983b/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:11:39.938422 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef65c2d3-14d6-4574-9229-e3a8990f5f7c admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:11:39.943114 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-ef65c2d3-14d6-4574-9229-e3a8990f5f7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:11:40.113527 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ca00d-5f8d-4385-aab4-1f091e24dc2a admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:11:40.116372 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-074ca00d-5f8d-4385-aab4-1f091e24dc2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:11:40.152915 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef65c2d3-14d6-4574-9229-e3a8990f5f7c admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:11:40.195975 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:11:40.242655 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ca00d-5f8d-4385-aab4-1f091e24dc2a admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:11:40.259425 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9859b79-fa72-4cf7-b04c-5a450857c458 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02941c6ad2d54e48b1114d9e030c66c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:11:40.260374 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9859b79-fa72-4cf7-b04c-5a450857c458 service placement] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:40.260887 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9859b79-fa72-4cf7-b04c-5a450857c458 service placement] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:40.261581 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9859b79-fa72-4cf7-b04c-5a450857c458 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:40.265976 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-f9859b79-fa72-4cf7-b04c-5a450857c458 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:11:40.278219 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd0f63-83ea-4938-afd6-ab871d5b9ff7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02941c6ad2d54e48b1114d9e030c66c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:11:40.279467 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd0f63-83ea-4938-afd6-ab871d5b9ff7 service placement] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:40.280128 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd0f63-83ea-4938-afd6-ab871d5b9ff7 service placement] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:40.280843 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd0f63-83ea-4938-afd6-ab871d5b9ff7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:40.287465 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9859b79-fa72-4cf7-b04c-5a450857c458 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:11:40.290237 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9859b79-fa72-4cf7-b04c-5a450857c458 service placement] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:11:40.303032 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcd0f63-83ea-4938-afd6-ab871d5b9ff7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:11:40.305557 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcd0f63-83ea-4938-afd6-ab871d5b9ff7 service placement] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:11:40.310994 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1149/2317] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:11:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 22:11:40.350463 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1169/2318] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:11:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:11:42.270807 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45abb5b8-aae5-4d84-863b-1f89586ce674 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:42.272306 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45abb5b8-aae5-4d84-863b-1f89586ce674 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:42.272819 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45abb5b8-aae5-4d84-863b-1f89586ce674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:42.330412 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-45abb5b8-aae5-4d84-863b-1f89586ce674 None None] MFA Rules not processed for user `df8302231ba9470c99b27bd0048e983b`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:11:42.333073 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-45abb5b8-aae5-4d84-863b-1f89586ce674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:11:42.401752 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:11:42.489200 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1150/2319] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:11:42 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:11:42.504952 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd0f63-83ea-4938-afd6-ab871d5b9ff7 service placement] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:11:42.506118 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-1bcd0f63-83ea-4938-afd6-ab871d5b9ff7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:11:42.671091 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd0f63-83ea-4938-afd6-ab871d5b9ff7 service placement] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:11:42.683787 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383a1066-e285-4638-8f3d-138636dcd704 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:11:42.690237 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a1066-e285-4638-8f3d-138636dcd704 service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:42.690565 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a1066-e285-4638-8f3d-138636dcd704 service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:42.690855 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a1066-e285-4638-8f3d-138636dcd704 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:42.791223 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383a1066-e285-4638-8f3d-138636dcd704 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:11:42.792915 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383a1066-e285-4638-8f3d-138636dcd704 service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:11:42.802884 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1170/2320] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:11:42 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:11:44.440963 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a173ff5-9da4-42fe-8e68-64b7be56d304 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:44.441477 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a173ff5-9da4-42fe-8e68-64b7be56d304 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:44.441734 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a173ff5-9da4-42fe-8e68-64b7be56d304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:44.461161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-1a173ff5-9da4-42fe-8e68-64b7be56d304 None None] MFA Rules not processed for user `7e8c46b1820a4ce5ac853ab1cc88fc30`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:11:44.464757 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-1a173ff5-9da4-42fe-8e68-64b7be56d304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:11:44.477654 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1151/2321] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:11:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:11:45.695376 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fbba52-33f0-4aca-9dfd-51658e9d0c51 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:45.695376 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fbba52-33f0-4aca-9dfd-51658e9d0c51 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:45.696043 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fbba52-33f0-4aca-9dfd-51658e9d0c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:45.719250 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-a6fbba52-33f0-4aca-9dfd-51658e9d0c51 None None] MFA Rules not processed for user `0e68eff6a08d4193b3167d47f4e1f97c`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:11:45.720712 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a6fbba52-33f0-4aca-9dfd-51658e9d0c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:11:45.774552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:11:45.787394 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1171/2322] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:11:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:11:54.455460 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=18 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:11:55.025559 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a173ff5-9da4-42fe-8e68-64b7be56d304 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:11:55.088478 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a173ff5-9da4-42fe-8e68-64b7be56d304 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:11:55.092065 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e70a9-0f2c-48e2-a1d6-3371b8d06a98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:11:55.092851 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e70a9-0f2c-48e2-a1d6-3371b8d06a98 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:55.093234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e70a9-0f2c-48e2-a1d6-3371b8d06a98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:55.093518 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e70a9-0f2c-48e2-a1d6-3371b8d06a98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:55.095309 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-1e2e70a9-0f2c-48e2-a1d6-3371b8d06a98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:11:55.159236 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e70a9-0f2c-48e2-a1d6-3371b8d06a98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:11:55.160710 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e70a9-0f2c-48e2-a1d6-3371b8d06a98 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:11:55.170427 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1152/2323] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:11:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:11:55.779241 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:11:58.533467 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fbba52-33f0-4aca-9dfd-51658e9d0c51 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:11:58.535012 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a6fbba52-33f0-4aca-9dfd-51658e9d0c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:11:58.550682 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fbba52-33f0-4aca-9dfd-51658e9d0c51 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:11:58.556755 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904037d6-81a9-4243-b24e-b9479ab5b496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:11:58.557759 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904037d6-81a9-4243-b24e-b9479ab5b496 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:58.558039 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904037d6-81a9-4243-b24e-b9479ab5b496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:58.558312 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904037d6-81a9-4243-b24e-b9479ab5b496 admin admin] PATH_INFO: `/v3/users/c44c1788959e4b339c9cd5aad4318496` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:58.559557 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904037d6-81a9-4243-b24e-b9479ab5b496 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c44c1788959e4b339c9cd5aad4318496)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:11:58.562179 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904037d6-81a9-4243-b24e-b9479ab5b496 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:11:58.613012 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-904037d6-81a9-4243-b24e-b9479ab5b496 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c44c1788959e4b339c9cd5aad4318496'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:11:58.620079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-904037d6-81a9-4243-b24e-b9479ab5b496 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c44c1788959e4b339c9cd5aad4318496'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:11:58.632494 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-904037d6-81a9-4243-b24e-b9479ab5b496 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c44c1788959e4b339c9cd5aad4318496'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:11:58.633185 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-904037d6-81a9-4243-b24e-b9479ab5b496 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c44c1788959e4b339c9cd5aad4318496'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:11:58.633780 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-904037d6-81a9-4243-b24e-b9479ab5b496 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c44c1788959e4b339c9cd5aad4318496'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:11:58.646979 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1172/2324] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:11:58 2022] DELETE /identity/v3/users/c44c1788959e4b339c9cd5aad4318496 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:11:58.826679 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e70a9-0f2c-48e2-a1d6-3371b8d06a98 service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:11:58.837477 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e70a9-0f2c-48e2-a1d6-3371b8d06a98 service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:11:58.897822 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b616c-14a5-4898-8e17-6ada7318749c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:11:58.898687 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b616c-14a5-4898-8e17-6ada7318749c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:58.898971 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b616c-14a5-4898-8e17-6ada7318749c admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:58.899249 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b616c-14a5-4898-8e17-6ada7318749c admin admin] PATH_INFO: `/v3/projects/0a4fc48a5efe4d8b974757a5bf7d947e` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:58.900480 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956b616c-14a5-4898-8e17-6ada7318749c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a4fc48a5efe4d8b974757a5bf7d947e)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:11:58.902391 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956b616c-14a5-4898-8e17-6ada7318749c admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:11:58.952772 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-956b616c-14a5-4898-8e17-6ada7318749c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a4fc48a5efe4d8b974757a5bf7d947e', 'request_id': 'req-956b616c-14a5-4898-8e17-6ada7318749c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:11:58.955212 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:11:58.960579 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-956b616c-14a5-4898-8e17-6ada7318749c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a4fc48a5efe4d8b974757a5bf7d947e', 'request_id': 'req-956b616c-14a5-4898-8e17-6ada7318749c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:11:58.969080 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-956b616c-14a5-4898-8e17-6ada7318749c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a4fc48a5efe4d8b974757a5bf7d947e', 'request_id': 'req-956b616c-14a5-4898-8e17-6ada7318749c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:11:58.979627 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-956b616c-14a5-4898-8e17-6ada7318749c admin admin] The token cache is being invalidate because project 0a4fc48a5efe4d8b974757a5bf7d947e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:11:58.979918 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-956b616c-14a5-4898-8e17-6ada7318749c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:11:58.980553 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-956b616c-14a5-4898-8e17-6ada7318749c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:11:58.982079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1153/2325] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:11:58 2022] DELETE /identity/v3/projects/0a4fc48a5efe4d8b974757a5bf7d947e => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:11:58.987792 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e64a7-5627-4c4c-a340-4628e684c49c None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:58.988107 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e64a7-5627-4c4c-a340-4628e684c49c None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:58.988399 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e64a7-5627-4c4c-a340-4628e684c49c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:59.006102 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-8c3e64a7-5627-4c4c-a340-4628e684c49c None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:11:59.007285 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-8c3e64a7-5627-4c4c-a340-4628e684c49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:11:59.064780 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1173/2326] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:11:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:11:59.073495 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e3adb6-c05f-4184-a47e-64598fe24bb0 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:59.073782 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e3adb6-c05f-4184-a47e-64598fe24bb0 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:59.074063 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e3adb6-c05f-4184-a47e-64598fe24bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:59.093156 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-26e3adb6-c05f-4184-a47e-64598fe24bb0 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:11:59.094610 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-26e3adb6-c05f-4184-a47e-64598fe24bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:11:59.102717 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1154/2327] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:11:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:11:59.106766 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e64a7-5627-4c4c-a340-4628e684c49c None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:11:59.117084 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e64a7-5627-4c4c-a340-4628e684c49c None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:11:59.121579 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee704ae6-bcf5-4db7-a3ba-d6b3a1ae7b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:11:59.122221 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee704ae6-bcf5-4db7-a3ba-d6b3a1ae7b85 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:59.122502 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee704ae6-bcf5-4db7-a3ba-d6b3a1ae7b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:59.122774 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee704ae6-bcf5-4db7-a3ba-d6b3a1ae7b85 admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:59.123528 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee704ae6-bcf5-4db7-a3ba-d6b3a1ae7b85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:11:59.123837 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee704ae6-bcf5-4db7-a3ba-d6b3a1ae7b85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:11:59.125414 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee704ae6-bcf5-4db7-a3ba-d6b3a1ae7b85 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:11:59.136649 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1174/2328] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:11:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:11:59.141074 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e3adb6-c05f-4184-a47e-64598fe24bb0 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:11:59.150076 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e3adb6-c05f-4184-a47e-64598fe24bb0 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:11:59.154727 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4c0592-e873-4a4a-90bf-4296f7174505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:11:59.156049 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c0592-e873-4a4a-90bf-4296f7174505 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:59.156337 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c0592-e873-4a4a-90bf-4296f7174505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:59.156617 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c0592-e873-4a4a-90bf-4296f7174505 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:59.157321 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4c0592-e873-4a4a-90bf-4296f7174505 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:11:59.159269 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4c0592-e873-4a4a-90bf-4296f7174505 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:11:59.187794 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1155/2329] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:11:59 2022] POST /identity/v3/projects => generated 401 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:11:59.191795 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee704ae6-bcf5-4db7-a3ba-d6b3a1ae7b85 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:11:59.201683 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee704ae6-bcf5-4db7-a3ba-d6b3a1ae7b85 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:11:59.263572 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beafcf1d-169e-4860-8c4d-da640f8381c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:11:59.264246 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beafcf1d-169e-4860-8c4d-da640f8381c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:59.264664 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beafcf1d-169e-4860-8c4d-da640f8381c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:59.265171 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beafcf1d-169e-4860-8c4d-da640f8381c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:59.265913 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beafcf1d-169e-4860-8c4d-da640f8381c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:11:59.267871 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beafcf1d-169e-4860-8c4d-da640f8381c3 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:11:59.274512 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:11:59.289050 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1175/2330] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:11:59 2022] POST /identity/v3/users => generated 301 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:11:59.293337 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4c0592-e873-4a4a-90bf-4296f7174505 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:11:59.304532 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4c0592-e873-4a4a-90bf-4296f7174505 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:11:59.309296 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae0a5da-c0ba-4be2-93aa-56241969e7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:11:59.309991 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0a5da-c0ba-4be2-93aa-56241969e7b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:59.310281 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0a5da-c0ba-4be2-93aa-56241969e7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:59.310559 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0a5da-c0ba-4be2-93aa-56241969e7b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:59.311346 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae0a5da-c0ba-4be2-93aa-56241969e7b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:11:59.313021 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae0a5da-c0ba-4be2-93aa-56241969e7b4 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:11:59.336414 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1156/2331] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:11:59 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:11:59.340679 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beafcf1d-169e-4860-8c4d-da640f8381c3 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:11:59.352234 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beafcf1d-169e-4860-8c4d-da640f8381c3 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:11:59.357000 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b5fa1e-d1a7-4084-a279-22e691c497ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:11:59.357615 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b5fa1e-d1a7-4084-a279-22e691c497ec admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:59.357911 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b5fa1e-d1a7-4084-a279-22e691c497ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:59.358261 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b5fa1e-d1a7-4084-a279-22e691c497ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:59.359478 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b5fa1e-d1a7-4084-a279-22e691c497ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:11:59.361483 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b5fa1e-d1a7-4084-a279-22e691c497ec admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:11:59.372212 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1176/2332] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:11:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 22:11:59.376955 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae0a5da-c0ba-4be2-93aa-56241969e7b4 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:11:59.387385 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae0a5da-c0ba-4be2-93aa-56241969e7b4 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:11:59.393103 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4041abdf-175d-47e6-be41-067c5f3c704d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:11:59.394004 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041abdf-175d-47e6-be41-067c5f3c704d admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:59.394318 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041abdf-175d-47e6-be41-067c5f3c704d admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:59.394646 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041abdf-175d-47e6-be41-067c5f3c704d admin admin] PATH_INFO: `/v3/projects/3372d3d2c5a94a018e7861d4493b07bc/users/0a2b37e45d984964b0e7f5da100905c8/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:59.411014 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4041abdf-175d-47e6-be41-067c5f3c704d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3372d3d2c5a94a018e7861d4493b07bc, user_id=0a2b37e45d984964b0e7f5da100905c8, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:11:59.413689 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4041abdf-175d-47e6-be41-067c5f3c704d admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:11:59.428774 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1157/2333] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:11:59 2022] PUT /identity/v3/projects/3372d3d2c5a94a018e7861d4493b07bc/users/0a2b37e45d984964b0e7f5da100905c8/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:11:59.439206 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee50968e-3f69-4add-841e-33937018749f None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:59.439537 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee50968e-3f69-4add-841e-33937018749f None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:59.439828 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee50968e-3f69-4add-841e-33937018749f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:59.466977 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-ee50968e-3f69-4add-841e-33937018749f None None] MFA Rules not processed for user `0a2b37e45d984964b0e7f5da100905c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:11:59.468801 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-ee50968e-3f69-4add-841e-33937018749f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:11:59.530719 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1177/2334] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:11:59 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:11:59.535491 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4041abdf-175d-47e6-be41-067c5f3c704d admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:11:59.598282 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4041abdf-175d-47e6-be41-067c5f3c704d admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:11:59.603392 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f61343e-2883-4c4f-902d-57e1062d699a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:11:59.604071 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f61343e-2883-4c4f-902d-57e1062d699a admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:59.604355 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f61343e-2883-4c4f-902d-57e1062d699a admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:59.604627 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f61343e-2883-4c4f-902d-57e1062d699a admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:59.605327 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f61343e-2883-4c4f-902d-57e1062d699a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:11:59.607297 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f61343e-2883-4c4f-902d-57e1062d699a admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:11:59.632964 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1158/2335] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:11:59 2022] POST /identity/v3/projects => generated 403 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:11:59.637059 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee50968e-3f69-4add-841e-33937018749f None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:11:59.648196 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee50968e-3f69-4add-841e-33937018749f None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:11:59.705961 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674cc07f-687b-4a61-8624-598a5e97825e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:11:59.706635 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674cc07f-687b-4a61-8624-598a5e97825e admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:59.706921 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674cc07f-687b-4a61-8624-598a5e97825e admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:59.707200 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674cc07f-687b-4a61-8624-598a5e97825e admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:59.707937 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674cc07f-687b-4a61-8624-598a5e97825e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:11:59.709988 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674cc07f-687b-4a61-8624-598a5e97825e admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:11:59.731445 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1178/2336] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:11:59 2022] POST /identity/v3/users => generated 308 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:11:59.736468 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f61343e-2883-4c4f-902d-57e1062d699a admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:11:59.750629 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f61343e-2883-4c4f-902d-57e1062d699a admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:11:59.756658 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeefa81-641c-4c5e-82a6-19d9d48d9749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:11:59.757343 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeefa81-641c-4c5e-82a6-19d9d48d9749 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:59.757681 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeefa81-641c-4c5e-82a6-19d9d48d9749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:59.757967 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeefa81-641c-4c5e-82a6-19d9d48d9749 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:59.758717 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeefa81-641c-4c5e-82a6-19d9d48d9749 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:11:59.760518 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeefa81-641c-4c5e-82a6-19d9d48d9749 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:11:59.783237 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1159/2337] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:11:59 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:11:59.787080 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674cc07f-687b-4a61-8624-598a5e97825e admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:11:59.797881 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674cc07f-687b-4a61-8624-598a5e97825e admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:11:59.802728 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0196d3c4-c76b-4d6d-884e-f48e7724286d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:11:59.803638 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196d3c4-c76b-4d6d-884e-f48e7724286d admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:59.803923 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196d3c4-c76b-4d6d-884e-f48e7724286d admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:59.804200 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196d3c4-c76b-4d6d-884e-f48e7724286d admin admin] PATH_INFO: `/v3/projects/df89d697f5fd484c8e9bb202f552c76c/users/f4926c7662f84670bf56bba7dadabcb6/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:59.821198 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0196d3c4-c76b-4d6d-884e-f48e7724286d admin admin] RBAC: Authorizing `identity:create_grant(project_id=df89d697f5fd484c8e9bb202f552c76c, user_id=f4926c7662f84670bf56bba7dadabcb6, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:11:59.823884 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0196d3c4-c76b-4d6d-884e-f48e7724286d admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:11:59.837657 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1179/2338] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:11:59 2022] PUT /identity/v3/projects/df89d697f5fd484c8e9bb202f552c76c/users/f4926c7662f84670bf56bba7dadabcb6/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:11:59.841651 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeefa81-641c-4c5e-82a6-19d9d48d9749 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:11:59.852284 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeefa81-641c-4c5e-82a6-19d9d48d9749 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:11:59.898734 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:11:59.909556 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9387dc1-aa3b-4024-a48a-d44afbbf8a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:11:59.910224 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9387dc1-aa3b-4024-a48a-d44afbbf8a63 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:59.910506 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9387dc1-aa3b-4024-a48a-d44afbbf8a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:59.910780 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9387dc1-aa3b-4024-a48a-d44afbbf8a63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:59.911535 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9387dc1-aa3b-4024-a48a-d44afbbf8a63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:11:59.913127 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9387dc1-aa3b-4024-a48a-d44afbbf8a63 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:11:59.923155 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1160/2339] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:11:59 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:11:59.927386 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0196d3c4-c76b-4d6d-884e-f48e7724286d admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:11:59.938310 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0196d3c4-c76b-4d6d-884e-f48e7724286d admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:11:59.942942 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ce4f67-6cdc-4495-b6ab-4410ebe4d4c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:11:59.943856 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce4f67-6cdc-4495-b6ab-4410ebe4d4c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:59.944138 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce4f67-6cdc-4495-b6ab-4410ebe4d4c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:59.944412 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce4f67-6cdc-4495-b6ab-4410ebe4d4c9 admin admin] PATH_INFO: `/v3/projects/df89d697f5fd484c8e9bb202f552c76c/users/f4926c7662f84670bf56bba7dadabcb6/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:59.946591 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ce4f67-6cdc-4495-b6ab-4410ebe4d4c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df89d697f5fd484c8e9bb202f552c76c, user_id=f4926c7662f84670bf56bba7dadabcb6, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:11:59.949257 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ce4f67-6cdc-4495-b6ab-4410ebe4d4c9 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:11:59.960130 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1180/2340] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:11:59 2022] PUT /identity/v3/projects/df89d697f5fd484c8e9bb202f552c76c/users/f4926c7662f84670bf56bba7dadabcb6/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:11:59.970070 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4506fb5-9f2a-461b-8682-0491c6ab17a3 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:11:59.970402 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4506fb5-9f2a-461b-8682-0491c6ab17a3 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:11:59.970717 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4506fb5-9f2a-461b-8682-0491c6ab17a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:11:59.996449 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-d4506fb5-9f2a-461b-8682-0491c6ab17a3 None None] MFA Rules not processed for user `f4926c7662f84670bf56bba7dadabcb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:11:59.997671 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-d4506fb5-9f2a-461b-8682-0491c6ab17a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:00.064033 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1161/2341] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:11:59 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:12:00.074567 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ce4f67-6cdc-4495-b6ab-4410ebe4d4c9 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:00.076312 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-05ce4f67-6cdc-4495-b6ab-4410ebe4d4c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:00.101317 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:12:00.138883 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ce4f67-6cdc-4495-b6ab-4410ebe4d4c9 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:00.142838 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8749a2d0-729f-482a-8c2c-110cd8e68060 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:00.143516 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8749a2d0-729f-482a-8c2c-110cd8e68060 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:00.143803 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8749a2d0-729f-482a-8c2c-110cd8e68060 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:00.144082 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8749a2d0-729f-482a-8c2c-110cd8e68060 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:00.153215 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8749a2d0-729f-482a-8c2c-110cd8e68060 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:00.154588 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8749a2d0-729f-482a-8c2c-110cd8e68060 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:00.164026 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1181/2342] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:12:00 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:12:00.434458 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4506fb5-9f2a-461b-8682-0491c6ab17a3 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:00.497518 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4506fb5-9f2a-461b-8682-0491c6ab17a3 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:00.502748 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557883a0-4949-4592-b7b0-6b7b96cb30cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:00.503670 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557883a0-4949-4592-b7b0-6b7b96cb30cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:00.503995 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557883a0-4949-4592-b7b0-6b7b96cb30cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:00.504297 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557883a0-4949-4592-b7b0-6b7b96cb30cb admin admin] PATH_INFO: `/v3/users/0a2b37e45d984964b0e7f5da100905c8` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:00.505493 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557883a0-4949-4592-b7b0-6b7b96cb30cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a2b37e45d984964b0e7f5da100905c8)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:00.507485 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557883a0-4949-4592-b7b0-6b7b96cb30cb admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:00.554444 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-557883a0-4949-4592-b7b0-6b7b96cb30cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a2b37e45d984964b0e7f5da100905c8'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:00.555106 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-557883a0-4949-4592-b7b0-6b7b96cb30cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a2b37e45d984964b0e7f5da100905c8'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:00.562906 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-557883a0-4949-4592-b7b0-6b7b96cb30cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a2b37e45d984964b0e7f5da100905c8'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:00.574828 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-557883a0-4949-4592-b7b0-6b7b96cb30cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a2b37e45d984964b0e7f5da100905c8'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:00.576841 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:12:00.584331 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-557883a0-4949-4592-b7b0-6b7b96cb30cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a2b37e45d984964b0e7f5da100905c8'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:00.590672 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1162/2343] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:12:00 2022] DELETE /identity/v3/users/0a2b37e45d984964b0e7f5da100905c8 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:12:00.595799 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca46c9-9a3d-4329-9503-4988a20a7eb0 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:00.596095 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca46c9-9a3d-4329-9503-4988a20a7eb0 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:00.596373 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca46c9-9a3d-4329-9503-4988a20a7eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:00.613632 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-ffca46c9-9a3d-4329-9503-4988a20a7eb0 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:12:00.614845 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-ffca46c9-9a3d-4329-9503-4988a20a7eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:00.673129 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1182/2344] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:12:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:12:00.682918 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557883a0-4949-4592-b7b0-6b7b96cb30cb admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:00.684334 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-557883a0-4949-4592-b7b0-6b7b96cb30cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:00.741153 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557883a0-4949-4592-b7b0-6b7b96cb30cb admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:00.744831 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1aee6-3ccf-48ff-9c09-86b9f72d9d8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:00.745487 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1aee6-3ccf-48ff-9c09-86b9f72d9d8d service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:00.745806 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1aee6-3ccf-48ff-9c09-86b9f72d9d8d service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:00.746117 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1aee6-3ccf-48ff-9c09-86b9f72d9d8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:00.756817 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da1aee6-3ccf-48ff-9c09-86b9f72d9d8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:00.758272 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da1aee6-3ccf-48ff-9c09-86b9f72d9d8d service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:00.768600 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1163/2345] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:12:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:12:01.076829 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca46c9-9a3d-4329-9503-4988a20a7eb0 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:01.078899 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-ffca46c9-9a3d-4329-9503-4988a20a7eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:01.091211 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca46c9-9a3d-4329-9503-4988a20a7eb0 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:01.096076 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca93f3c-80fe-400b-925d-1c44c0f14259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:01.096952 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca93f3c-80fe-400b-925d-1c44c0f14259 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:01.097236 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca93f3c-80fe-400b-925d-1c44c0f14259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:01.097515 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca93f3c-80fe-400b-925d-1c44c0f14259 admin admin] PATH_INFO: `/v3/projects/3372d3d2c5a94a018e7861d4493b07bc` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:01.099375 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca93f3c-80fe-400b-925d-1c44c0f14259 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3372d3d2c5a94a018e7861d4493b07bc)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:01.100856 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca93f3c-80fe-400b-925d-1c44c0f14259 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:01.155841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3ca93f3c-80fe-400b-925d-1c44c0f14259 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3372d3d2c5a94a018e7861d4493b07bc', 'request_id': 'req-3ca93f3c-80fe-400b-925d-1c44c0f14259'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:01.166869 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3ca93f3c-80fe-400b-925d-1c44c0f14259 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3372d3d2c5a94a018e7861d4493b07bc', 'request_id': 'req-3ca93f3c-80fe-400b-925d-1c44c0f14259'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:01.175790 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3ca93f3c-80fe-400b-925d-1c44c0f14259 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3372d3d2c5a94a018e7861d4493b07bc', 'request_id': 'req-3ca93f3c-80fe-400b-925d-1c44c0f14259'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:01.187361 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3ca93f3c-80fe-400b-925d-1c44c0f14259 admin admin] The token cache is being invalidate because project 3372d3d2c5a94a018e7861d4493b07bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:12:01.187710 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3ca93f3c-80fe-400b-925d-1c44c0f14259 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:01.188360 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3ca93f3c-80fe-400b-925d-1c44c0f14259 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:01.189890 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1183/2346] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:12:01 2022] DELETE /identity/v3/projects/3372d3d2c5a94a018e7861d4493b07bc => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:12:01.193955 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1aee6-3ccf-48ff-9c09-86b9f72d9d8d service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:01.204199 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1aee6-3ccf-48ff-9c09-86b9f72d9d8d service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:01.262092 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc82832-ce8c-4e50-92ef-e06f3081970a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:01.262949 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc82832-ce8c-4e50-92ef-e06f3081970a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:01.263249 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc82832-ce8c-4e50-92ef-e06f3081970a admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:01.263590 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc82832-ce8c-4e50-92ef-e06f3081970a admin admin] PATH_INFO: `/v3/users/f4926c7662f84670bf56bba7dadabcb6` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:01.265376 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc82832-ce8c-4e50-92ef-e06f3081970a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4926c7662f84670bf56bba7dadabcb6)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:01.266927 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc82832-ce8c-4e50-92ef-e06f3081970a admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:01.315036 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-bdc82832-ce8c-4e50-92ef-e06f3081970a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4926c7662f84670bf56bba7dadabcb6'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:01.315738 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-bdc82832-ce8c-4e50-92ef-e06f3081970a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4926c7662f84670bf56bba7dadabcb6'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:01.324138 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-bdc82832-ce8c-4e50-92ef-e06f3081970a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4926c7662f84670bf56bba7dadabcb6'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:01.337551 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-bdc82832-ce8c-4e50-92ef-e06f3081970a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4926c7662f84670bf56bba7dadabcb6'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:01.344680 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-bdc82832-ce8c-4e50-92ef-e06f3081970a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4926c7662f84670bf56bba7dadabcb6'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:01.352640 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1164/2347] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:12:01 2022] DELETE /identity/v3/users/f4926c7662f84670bf56bba7dadabcb6 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:12:01.828529 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca93f3c-80fe-400b-925d-1c44c0f14259 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:01.829965 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-3ca93f3c-80fe-400b-925d-1c44c0f14259 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:01.842976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca93f3c-80fe-400b-925d-1c44c0f14259 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:01.888917 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:12:01.915423 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ffb06b-99e4-4b98-b6e4-e22bd182af45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:01.916318 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ffb06b-99e4-4b98-b6e4-e22bd182af45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:01.916602 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ffb06b-99e4-4b98-b6e4-e22bd182af45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:01.916873 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ffb06b-99e4-4b98-b6e4-e22bd182af45 admin admin] PATH_INFO: `/v3/projects/df89d697f5fd484c8e9bb202f552c76c` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:01.918138 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ffb06b-99e4-4b98-b6e4-e22bd182af45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df89d697f5fd484c8e9bb202f552c76c)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:01.920128 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ffb06b-99e4-4b98-b6e4-e22bd182af45 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:01.980913 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-76ffb06b-99e4-4b98-b6e4-e22bd182af45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df89d697f5fd484c8e9bb202f552c76c', 'request_id': 'req-76ffb06b-99e4-4b98-b6e4-e22bd182af45'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:01.989726 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-76ffb06b-99e4-4b98-b6e4-e22bd182af45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df89d697f5fd484c8e9bb202f552c76c', 'request_id': 'req-76ffb06b-99e4-4b98-b6e4-e22bd182af45'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:01.995829 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-76ffb06b-99e4-4b98-b6e4-e22bd182af45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df89d697f5fd484c8e9bb202f552c76c', 'request_id': 'req-76ffb06b-99e4-4b98-b6e4-e22bd182af45'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:02.007974 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-76ffb06b-99e4-4b98-b6e4-e22bd182af45 admin admin] The token cache is being invalidate because project df89d697f5fd484c8e9bb202f552c76c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:12:02.008264 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-76ffb06b-99e4-4b98-b6e4-e22bd182af45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:02.008924 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-76ffb06b-99e4-4b98-b6e4-e22bd182af45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:02.010483 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1184/2348] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:12:01 2022] DELETE /identity/v3/projects/df89d697f5fd484c8e9bb202f552c76c => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:12:02.016320 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9b5c6-7dd8-4978-9732-2eca9fff4d3a None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:02.016628 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9b5c6-7dd8-4978-9732-2eca9fff4d3a None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:02.016943 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9b5c6-7dd8-4978-9732-2eca9fff4d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:02.035743 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-45a9b5c6-7dd8-4978-9732-2eca9fff4d3a None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:12:02.037106 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-45a9b5c6-7dd8-4978-9732-2eca9fff4d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:02.046746 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:12:02.104091 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1165/2349] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:12:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:12:02.113906 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c49d74-826f-4d56-90b0-ada1655436e1 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:02.114196 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c49d74-826f-4d56-90b0-ada1655436e1 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:02.114479 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c49d74-826f-4d56-90b0-ada1655436e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:02.134201 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-79c49d74-826f-4d56-90b0-ada1655436e1 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:12:02.135463 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-79c49d74-826f-4d56-90b0-ada1655436e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:02.143715 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1185/2350] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:12:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:12:02.148017 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9b5c6-7dd8-4978-9732-2eca9fff4d3a None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:02.159173 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9b5c6-7dd8-4978-9732-2eca9fff4d3a None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:02.164314 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb362625-2c8f-47e0-a712-9658bd2c93fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:02.165085 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb362625-2c8f-47e0-a712-9658bd2c93fc admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:02.165488 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb362625-2c8f-47e0-a712-9658bd2c93fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:02.165894 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb362625-2c8f-47e0-a712-9658bd2c93fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:02.166895 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb362625-2c8f-47e0-a712-9658bd2c93fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:12:02.167345 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb362625-2c8f-47e0-a712-9658bd2c93fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:02.169067 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb362625-2c8f-47e0-a712-9658bd2c93fc admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:02.185583 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1166/2351] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:12:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:12:02.186796 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c49d74-826f-4d56-90b0-ada1655436e1 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:02.196765 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c49d74-826f-4d56-90b0-ada1655436e1 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:02.201399 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82f2010-7cab-48a0-b446-f33afc10fcb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:02.202095 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f2010-7cab-48a0-b446-f33afc10fcb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:02.202375 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f2010-7cab-48a0-b446-f33afc10fcb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:02.202804 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f2010-7cab-48a0-b446-f33afc10fcb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:02.203825 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82f2010-7cab-48a0-b446-f33afc10fcb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:02.205769 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82f2010-7cab-48a0-b446-f33afc10fcb4 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:02.232907 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1186/2352] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:12:02 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:12:02.237961 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb362625-2c8f-47e0-a712-9658bd2c93fc admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:02.248224 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb362625-2c8f-47e0-a712-9658bd2c93fc admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:02.309795 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9384384f-1966-4ee5-9ed0-ecf51a14a6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:02.310441 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384384f-1966-4ee5-9ed0-ecf51a14a6ae admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:02.310723 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384384f-1966-4ee5-9ed0-ecf51a14a6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:02.310997 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384384f-1966-4ee5-9ed0-ecf51a14a6ae admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:02.312049 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9384384f-1966-4ee5-9ed0-ecf51a14a6ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:02.313985 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9384384f-1966-4ee5-9ed0-ecf51a14a6ae admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:02.336104 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1167/2353] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:12:02 2022] POST /identity/v3/users => generated 283 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:12:02.340500 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82f2010-7cab-48a0-b446-f33afc10fcb4 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:02.351558 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82f2010-7cab-48a0-b446-f33afc10fcb4 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:02.356793 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5b5b7-9a92-4b7c-8b0d-4658737c6556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:02.357320 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5b5b7-9a92-4b7c-8b0d-4658737c6556 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:02.357716 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5b5b7-9a92-4b7c-8b0d-4658737c6556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:02.357858 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5b5b7-9a92-4b7c-8b0d-4658737c6556 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:02.359088 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5b5b7-9a92-4b7c-8b0d-4658737c6556 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:02.360898 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5b5b7-9a92-4b7c-8b0d-4658737c6556 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:02.372011 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1187/2354] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:12:02.376394 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9384384f-1966-4ee5-9ed0-ecf51a14a6ae admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:02.386480 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9384384f-1966-4ee5-9ed0-ecf51a14a6ae admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:02.393008 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f3bb45-145c-4220-a841-2c715343c3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:02.394336 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f3bb45-145c-4220-a841-2c715343c3ab admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:02.394783 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f3bb45-145c-4220-a841-2c715343c3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:02.395217 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f3bb45-145c-4220-a841-2c715343c3ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:02.396157 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f3bb45-145c-4220-a841-2c715343c3ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:02.397906 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f3bb45-145c-4220-a841-2c715343c3ab admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:02.411674 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1168/2355] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:12:02.416384 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5b5b7-9a92-4b7c-8b0d-4658737c6556 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:02.428107 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5b5b7-9a92-4b7c-8b0d-4658737c6556 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:02.434986 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750aac70-88fe-439d-af49-ed007d2aab57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:02.435915 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750aac70-88fe-439d-af49-ed007d2aab57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:02.436235 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750aac70-88fe-439d-af49-ed007d2aab57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:02.436510 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750aac70-88fe-439d-af49-ed007d2aab57 admin admin] PATH_INFO: `/v3/projects/d4681e1459bf4d04b4afb2802f8f9e93/users/af9ed071a5e44a0497274864adfb9927/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:02.452247 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750aac70-88fe-439d-af49-ed007d2aab57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4681e1459bf4d04b4afb2802f8f9e93, user_id=af9ed071a5e44a0497274864adfb9927, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:02.455266 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750aac70-88fe-439d-af49-ed007d2aab57 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:02.470101 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1188/2356] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:12:02 2022] PUT /identity/v3/projects/d4681e1459bf4d04b4afb2802f8f9e93/users/af9ed071a5e44a0497274864adfb9927/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:12:02.480830 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6daf68-7594-4c93-8b54-26cc37427a6c None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:02.481120 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6daf68-7594-4c93-8b54-26cc37427a6c None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:02.481404 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6daf68-7594-4c93-8b54-26cc37427a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:02.509598 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-6f6daf68-7594-4c93-8b54-26cc37427a6c None None] MFA Rules not processed for user `af9ed071a5e44a0497274864adfb9927`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:12:02.510835 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-6f6daf68-7594-4c93-8b54-26cc37427a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:02.593877 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1169/2357] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:12:02 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:12:02.595055 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750aac70-88fe-439d-af49-ed007d2aab57 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:02.665383 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750aac70-88fe-439d-af49-ed007d2aab57 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:02.670142 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a474d38b-e37f-4218-bb6c-5552ca752ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:02.670956 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a474d38b-e37f-4218-bb6c-5552ca752ec7 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:02.671236 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a474d38b-e37f-4218-bb6c-5552ca752ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:02.671651 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a474d38b-e37f-4218-bb6c-5552ca752ec7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:02.672349 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a474d38b-e37f-4218-bb6c-5552ca752ec7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:02.674267 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a474d38b-e37f-4218-bb6c-5552ca752ec7 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:02.696452 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:12:02.708562 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1189/2358] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:12:02 2022] POST /identity/v3/projects => generated 365 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:12:02.709282 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6daf68-7594-4c93-8b54-26cc37427a6c None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:02.721950 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6daf68-7594-4c93-8b54-26cc37427a6c None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:02.728102 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:12:02.794696 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de44e52-a0b6-40a5-b0fb-c08e5429925f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:02.795553 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de44e52-a0b6-40a5-b0fb-c08e5429925f admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:02.795937 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de44e52-a0b6-40a5-b0fb-c08e5429925f admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:02.796285 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de44e52-a0b6-40a5-b0fb-c08e5429925f admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:02.797058 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de44e52-a0b6-40a5-b0fb-c08e5429925f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:02.799204 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de44e52-a0b6-40a5-b0fb-c08e5429925f admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:02.823785 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1170/2359] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:12:02 2022] POST /identity/v3/users => generated 289 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:12:02.825209 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a474d38b-e37f-4218-bb6c-5552ca752ec7 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:02.836051 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a474d38b-e37f-4218-bb6c-5552ca752ec7 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:02.840933 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf92b8d9-195f-4b8c-aabc-017294b33693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:02.841661 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92b8d9-195f-4b8c-aabc-017294b33693 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:02.842046 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92b8d9-195f-4b8c-aabc-017294b33693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:02.843133 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92b8d9-195f-4b8c-aabc-017294b33693 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:02.844069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf92b8d9-195f-4b8c-aabc-017294b33693 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:02.845863 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf92b8d9-195f-4b8c-aabc-017294b33693 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:02.861241 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1190/2360] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 22:12:02.862591 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de44e52-a0b6-40a5-b0fb-c08e5429925f admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:02.873252 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de44e52-a0b6-40a5-b0fb-c08e5429925f admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:02.879353 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd70dd-bcce-4baf-a30f-8488455eba22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:02.880502 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd70dd-bcce-4baf-a30f-8488455eba22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:02.880670 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd70dd-bcce-4baf-a30f-8488455eba22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:02.880998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd70dd-bcce-4baf-a30f-8488455eba22 admin admin] PATH_INFO: `/v3/projects/8ecdb6227eae47fe82e4d89b35289553/users/565514dcfe374c1398b77a7bf760974a/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:02.897549 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efd70dd-bcce-4baf-a30f-8488455eba22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ecdb6227eae47fe82e4d89b35289553, user_id=565514dcfe374c1398b77a7bf760974a, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:02.900356 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efd70dd-bcce-4baf-a30f-8488455eba22 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:02.916520 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1171/2361] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:12:02 2022] PUT /identity/v3/projects/8ecdb6227eae47fe82e4d89b35289553/users/565514dcfe374c1398b77a7bf760974a/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:12:02.921350 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf92b8d9-195f-4b8c-aabc-017294b33693 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:02.932623 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf92b8d9-195f-4b8c-aabc-017294b33693 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:03.013552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05163f06-28e5-481e-9604-06cdd7f0f77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:03.014310 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05163f06-28e5-481e-9604-06cdd7f0f77f admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:03.014656 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05163f06-28e5-481e-9604-06cdd7f0f77f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:03.015018 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05163f06-28e5-481e-9604-06cdd7f0f77f admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:03.015814 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05163f06-28e5-481e-9604-06cdd7f0f77f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:03.017490 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05163f06-28e5-481e-9604-06cdd7f0f77f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:03.028561 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1191/2362] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 22:12:03.033298 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd70dd-bcce-4baf-a30f-8488455eba22 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:03.046210 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd70dd-bcce-4baf-a30f-8488455eba22 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:03.051804 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b50660b-4a97-4962-8f76-116a12e468d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:03.052756 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50660b-4a97-4962-8f76-116a12e468d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:03.053041 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50660b-4a97-4962-8f76-116a12e468d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:03.053366 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50660b-4a97-4962-8f76-116a12e468d2 admin admin] PATH_INFO: `/v3/projects/8ecdb6227eae47fe82e4d89b35289553/users/565514dcfe374c1398b77a7bf760974a/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:03.055846 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b50660b-4a97-4962-8f76-116a12e468d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ecdb6227eae47fe82e4d89b35289553, user_id=565514dcfe374c1398b77a7bf760974a, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:03.058493 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b50660b-4a97-4962-8f76-116a12e468d2 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:03.069440 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1172/2363] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:12:03 2022] PUT /identity/v3/projects/8ecdb6227eae47fe82e4d89b35289553/users/565514dcfe374c1398b77a7bf760974a/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:12:03.081408 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5740c37-5e79-4236-b41b-3ad6961d9182 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:03.081805 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5740c37-5e79-4236-b41b-3ad6961d9182 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:03.082146 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5740c37-5e79-4236-b41b-3ad6961d9182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:03.109414 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-d5740c37-5e79-4236-b41b-3ad6961d9182 None None] MFA Rules not processed for user `565514dcfe374c1398b77a7bf760974a`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:12:03.110634 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d5740c37-5e79-4236-b41b-3ad6961d9182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:03.189141 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1192/2364] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:12:03 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:12:03.200223 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b50660b-4a97-4962-8f76-116a12e468d2 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:03.202902 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-2b50660b-4a97-4962-8f76-116a12e468d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:03.263819 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b50660b-4a97-4962-8f76-116a12e468d2 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:03.268432 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d2e5df-7d41-46d3-8a44-1d8c8009f4c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:03.269075 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2e5df-7d41-46d3-8a44-1d8c8009f4c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:03.269361 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2e5df-7d41-46d3-8a44-1d8c8009f4c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:03.269638 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2e5df-7d41-46d3-8a44-1d8c8009f4c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:03.279502 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d2e5df-7d41-46d3-8a44-1d8c8009f4c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:03.281885 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d2e5df-7d41-46d3-8a44-1d8c8009f4c8 service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:03.292050 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1173/2365] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:12:03 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:12:03.367176 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5740c37-5e79-4236-b41b-3ad6961d9182 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:03.377299 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5740c37-5e79-4236-b41b-3ad6961d9182 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:03.381019 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc31824-f8f4-4d3d-a9af-8571393542df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:03.381651 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc31824-f8f4-4d3d-a9af-8571393542df service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:03.381930 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc31824-f8f4-4d3d-a9af-8571393542df service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:03.382204 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc31824-f8f4-4d3d-a9af-8571393542df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:03.439964 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc31824-f8f4-4d3d-a9af-8571393542df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:03.441402 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc31824-f8f4-4d3d-a9af-8571393542df service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:03.450525 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1193/2366] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:12:03 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:12:13.287479 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=66,keystone:INSERT=5 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:12:13.446201 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=97 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:12:17.018140 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d2e5df-7d41-46d3-8a44-1d8c8009f4c8 service cinder] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:17.082295 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d2e5df-7d41-46d3-8a44-1d8c8009f4c8 service cinder] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:17.086746 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d97a73-7f46-45b3-9996-997f7d958340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:17.087639 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d97a73-7f46-45b3-9996-997f7d958340 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:17.087920 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d97a73-7f46-45b3-9996-997f7d958340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:17.088191 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d97a73-7f46-45b3-9996-997f7d958340 admin admin] PATH_INFO: `/v3/users/af9ed071a5e44a0497274864adfb9927` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:17.089363 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d97a73-7f46-45b3-9996-997f7d958340 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af9ed071a5e44a0497274864adfb9927)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:17.091241 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d97a73-7f46-45b3-9996-997f7d958340 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:17.136342 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-16d97a73-7f46-45b3-9996-997f7d958340 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af9ed071a5e44a0497274864adfb9927'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:17.136934 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-16d97a73-7f46-45b3-9996-997f7d958340 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af9ed071a5e44a0497274864adfb9927'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:17.145143 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-16d97a73-7f46-45b3-9996-997f7d958340 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af9ed071a5e44a0497274864adfb9927'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:17.157155 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-16d97a73-7f46-45b3-9996-997f7d958340 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af9ed071a5e44a0497274864adfb9927'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:17.163964 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-16d97a73-7f46-45b3-9996-997f7d958340 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af9ed071a5e44a0497274864adfb9927'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:17.169833 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1174/2367] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:12:17 2022] DELETE /identity/v3/users/af9ed071a5e44a0497274864adfb9927 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:12:17.174503 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa8cf0-d817-4f25-a5ea-0777648707f8 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:17.174928 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa8cf0-d817-4f25-a5ea-0777648707f8 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:17.175124 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa8cf0-d817-4f25-a5ea-0777648707f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:17.193347 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-95aa8cf0-d817-4f25-a5ea-0777648707f8 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:12:17.194543 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-95aa8cf0-d817-4f25-a5ea-0777648707f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:17.256541 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1194/2368] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:12:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:12:17.268623 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d97a73-7f46-45b3-9996-997f7d958340 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:17.270031 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-16d97a73-7f46-45b3-9996-997f7d958340 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:17.328590 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d97a73-7f46-45b3-9996-997f7d958340 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:17.332409 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbfe754-c7b9-4dc3-9249-70187fb68530 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:17.333218 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbfe754-c7b9-4dc3-9249-70187fb68530 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:17.333502 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbfe754-c7b9-4dc3-9249-70187fb68530 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:17.333779 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbfe754-c7b9-4dc3-9249-70187fb68530 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:17.343576 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbfe754-c7b9-4dc3-9249-70187fb68530 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:17.344997 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbfe754-c7b9-4dc3-9249-70187fb68530 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:17.355006 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1175/2369] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:12:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:12:17.631313 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aa8cf0-d817-4f25-a5ea-0777648707f8 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:17.632772 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-95aa8cf0-d817-4f25-a5ea-0777648707f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:17.645221 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aa8cf0-d817-4f25-a5ea-0777648707f8 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:17.649948 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072cb153-d0c3-4d4b-ae6a-e7c215fdd9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:17.650879 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072cb153-d0c3-4d4b-ae6a-e7c215fdd9d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:17.651164 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072cb153-d0c3-4d4b-ae6a-e7c215fdd9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:17.651473 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072cb153-d0c3-4d4b-ae6a-e7c215fdd9d4 admin admin] PATH_INFO: `/v3/projects/d4681e1459bf4d04b4afb2802f8f9e93` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:17.653091 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072cb153-d0c3-4d4b-ae6a-e7c215fdd9d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4681e1459bf4d04b4afb2802f8f9e93)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:17.655070 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072cb153-d0c3-4d4b-ae6a-e7c215fdd9d4 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:17.701523 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-072cb153-d0c3-4d4b-ae6a-e7c215fdd9d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4681e1459bf4d04b4afb2802f8f9e93', 'request_id': 'req-072cb153-d0c3-4d4b-ae6a-e7c215fdd9d4'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:17.709079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-072cb153-d0c3-4d4b-ae6a-e7c215fdd9d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4681e1459bf4d04b4afb2802f8f9e93', 'request_id': 'req-072cb153-d0c3-4d4b-ae6a-e7c215fdd9d4'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:17.716269 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-072cb153-d0c3-4d4b-ae6a-e7c215fdd9d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4681e1459bf4d04b4afb2802f8f9e93', 'request_id': 'req-072cb153-d0c3-4d4b-ae6a-e7c215fdd9d4'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:17.728598 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-072cb153-d0c3-4d4b-ae6a-e7c215fdd9d4 admin admin] The token cache is being invalidate because project d4681e1459bf4d04b4afb2802f8f9e93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:12:17.728889 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-072cb153-d0c3-4d4b-ae6a-e7c215fdd9d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:17.729548 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-072cb153-d0c3-4d4b-ae6a-e7c215fdd9d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:17.731215 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1195/2370] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:12:17 2022] DELETE /identity/v3/projects/d4681e1459bf4d04b4afb2802f8f9e93 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:12:17.735905 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbfe754-c7b9-4dc3-9249-70187fb68530 service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:17.745419 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbfe754-c7b9-4dc3-9249-70187fb68530 service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:17.764442 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:12:17.806713 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c267286-fee9-4143-8b7d-c3b4dcaf9363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:17.807620 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c267286-fee9-4143-8b7d-c3b4dcaf9363 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:17.807907 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c267286-fee9-4143-8b7d-c3b4dcaf9363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:17.808188 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c267286-fee9-4143-8b7d-c3b4dcaf9363 admin admin] PATH_INFO: `/v3/users/565514dcfe374c1398b77a7bf760974a` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:17.809382 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c267286-fee9-4143-8b7d-c3b4dcaf9363 admin admin] RBAC: Authorizing `identity:delete_user(user_id=565514dcfe374c1398b77a7bf760974a)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:17.811318 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c267286-fee9-4143-8b7d-c3b4dcaf9363 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:17.855749 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-4c267286-fee9-4143-8b7d-c3b4dcaf9363 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '565514dcfe374c1398b77a7bf760974a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:17.856406 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-4c267286-fee9-4143-8b7d-c3b4dcaf9363 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '565514dcfe374c1398b77a7bf760974a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:17.865400 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-4c267286-fee9-4143-8b7d-c3b4dcaf9363 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '565514dcfe374c1398b77a7bf760974a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:17.877149 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-4c267286-fee9-4143-8b7d-c3b4dcaf9363 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '565514dcfe374c1398b77a7bf760974a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:17.883925 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-4c267286-fee9-4143-8b7d-c3b4dcaf9363 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '565514dcfe374c1398b77a7bf760974a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:17.889759 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1176/2371] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:12:17 2022] DELETE /identity/v3/users/565514dcfe374c1398b77a7bf760974a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:12:18.163305 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072cb153-d0c3-4d4b-ae6a-e7c215fdd9d4 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:18.164834 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-072cb153-d0c3-4d4b-ae6a-e7c215fdd9d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:18.178596 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072cb153-d0c3-4d4b-ae6a-e7c215fdd9d4 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:18.237412 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c54b215-2e7c-4452-80ea-45e3f76d6ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:18.238298 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54b215-2e7c-4452-80ea-45e3f76d6ba6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:18.238576 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54b215-2e7c-4452-80ea-45e3f76d6ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:18.238846 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54b215-2e7c-4452-80ea-45e3f76d6ba6 admin admin] PATH_INFO: `/v3/projects/8ecdb6227eae47fe82e4d89b35289553` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:18.240035 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c54b215-2e7c-4452-80ea-45e3f76d6ba6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ecdb6227eae47fe82e4d89b35289553)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:18.241965 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c54b215-2e7c-4452-80ea-45e3f76d6ba6 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:18.270818 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:12:18.288547 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-4c54b215-2e7c-4452-80ea-45e3f76d6ba6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ecdb6227eae47fe82e4d89b35289553', 'request_id': 'req-4c54b215-2e7c-4452-80ea-45e3f76d6ba6'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:18.295693 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-4c54b215-2e7c-4452-80ea-45e3f76d6ba6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ecdb6227eae47fe82e4d89b35289553', 'request_id': 'req-4c54b215-2e7c-4452-80ea-45e3f76d6ba6'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:18.301446 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-4c54b215-2e7c-4452-80ea-45e3f76d6ba6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ecdb6227eae47fe82e4d89b35289553', 'request_id': 'req-4c54b215-2e7c-4452-80ea-45e3f76d6ba6'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:18.312339 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-4c54b215-2e7c-4452-80ea-45e3f76d6ba6 admin admin] The token cache is being invalidate because project 8ecdb6227eae47fe82e4d89b35289553 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:12:18.312627 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-4c54b215-2e7c-4452-80ea-45e3f76d6ba6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:18.313265 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-4c54b215-2e7c-4452-80ea-45e3f76d6ba6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:18.314780 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1196/2372] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:12:18 2022] DELETE /identity/v3/projects/8ecdb6227eae47fe82e4d89b35289553 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:12:18.320512 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8bb53-b398-41c7-965e-1f737e966a6a None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:18.320989 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8bb53-b398-41c7-965e-1f737e966a6a None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:18.321103 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8bb53-b398-41c7-965e-1f737e966a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:18.339153 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-dde8bb53-b398-41c7-965e-1f737e966a6a None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:12:18.341111 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-dde8bb53-b398-41c7-965e-1f737e966a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:18.410853 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1177/2373] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:12:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:12:18.415038 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0fe290-b21d-4f4e-b0c9-af0c863ef00a None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:18.415346 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0fe290-b21d-4f4e-b0c9-af0c863ef00a None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:18.415637 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0fe290-b21d-4f4e-b0c9-af0c863ef00a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:18.433876 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-4d0fe290-b21d-4f4e-b0c9-af0c863ef00a None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:12:18.435100 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-4d0fe290-b21d-4f4e-b0c9-af0c863ef00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:18.444253 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1197/2374] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:12:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:12:18.448639 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8bb53-b398-41c7-965e-1f737e966a6a None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:18.459283 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8bb53-b398-41c7-965e-1f737e966a6a None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:18.464219 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c304e436-9d52-4bf2-b55d-d5a9768fdd0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:18.464923 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c304e436-9d52-4bf2-b55d-d5a9768fdd0d admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:18.465157 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c304e436-9d52-4bf2-b55d-d5a9768fdd0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:18.465433 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c304e436-9d52-4bf2-b55d-d5a9768fdd0d admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:18.466161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c304e436-9d52-4bf2-b55d-d5a9768fdd0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:12:18.466472 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c304e436-9d52-4bf2-b55d-d5a9768fdd0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:18.468139 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c304e436-9d52-4bf2-b55d-d5a9768fdd0d admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:18.484391 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1178/2375] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:12:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:12:18.485676 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0fe290-b21d-4f4e-b0c9-af0c863ef00a None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:18.495031 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0fe290-b21d-4f4e-b0c9-af0c863ef00a None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:18.500227 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4b7517-ca02-4802-b00a-9a4bbd9f676f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:18.500886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b7517-ca02-4802-b00a-9a4bbd9f676f admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:18.501509 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b7517-ca02-4802-b00a-9a4bbd9f676f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:18.501509 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b7517-ca02-4802-b00a-9a4bbd9f676f admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:18.502164 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4b7517-ca02-4802-b00a-9a4bbd9f676f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:18.504124 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4b7517-ca02-4802-b00a-9a4bbd9f676f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:18.539225 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1198/2376] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:12:18 2022] POST /identity/v3/projects => generated 387 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:12:18.540384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c304e436-9d52-4bf2-b55d-d5a9768fdd0d admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:18.551339 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c304e436-9d52-4bf2-b55d-d5a9768fdd0d admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:18.576798 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:12:18.610598 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d10c328-d587-4b4d-b96b-a3f536d39a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:18.611213 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10c328-d587-4b4d-b96b-a3f536d39a0c admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:18.611528 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10c328-d587-4b4d-b96b-a3f536d39a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:18.611808 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10c328-d587-4b4d-b96b-a3f536d39a0c admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:18.612513 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d10c328-d587-4b4d-b96b-a3f536d39a0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:18.614412 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d10c328-d587-4b4d-b96b-a3f536d39a0c admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:18.633106 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1179/2377] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:12:18 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:12:18.637302 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4b7517-ca02-4802-b00a-9a4bbd9f676f admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:18.647576 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4b7517-ca02-4802-b00a-9a4bbd9f676f admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:18.652179 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5218e0-d82a-4c7c-b7f3-677db0726806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:18.652832 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5218e0-d82a-4c7c-b7f3-677db0726806 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:18.653110 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5218e0-d82a-4c7c-b7f3-677db0726806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:18.653384 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5218e0-d82a-4c7c-b7f3-677db0726806 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:18.654108 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5218e0-d82a-4c7c-b7f3-677db0726806 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:18.656328 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5218e0-d82a-4c7c-b7f3-677db0726806 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:18.666525 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1199/2378] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:12:18.670502 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d10c328-d587-4b4d-b96b-a3f536d39a0c admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:18.681360 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d10c328-d587-4b4d-b96b-a3f536d39a0c admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:18.685980 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f902254-8d10-4d48-b5c3-32bdd2eb801d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:18.686629 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f902254-8d10-4d48-b5c3-32bdd2eb801d admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:18.686919 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f902254-8d10-4d48-b5c3-32bdd2eb801d admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:18.687196 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f902254-8d10-4d48-b5c3-32bdd2eb801d admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:18.687959 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f902254-8d10-4d48-b5c3-32bdd2eb801d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:18.689542 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f902254-8d10-4d48-b5c3-32bdd2eb801d admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:18.699554 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1180/2379] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:12:18.703657 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5218e0-d82a-4c7c-b7f3-677db0726806 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:18.714801 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5218e0-d82a-4c7c-b7f3-677db0726806 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:18.720554 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f9fb5b-b2e3-43a5-a4e3-7fb7405c4bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:18.721537 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9fb5b-b2e3-43a5-a4e3-7fb7405c4bd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:18.721820 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9fb5b-b2e3-43a5-a4e3-7fb7405c4bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:18.722094 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9fb5b-b2e3-43a5-a4e3-7fb7405c4bd5 admin admin] PATH_INFO: `/v3/projects/d8842f728103446ba84c8efa9a9c7ae8/users/2b6d78cc0344441da955742b96cf91fd/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:18.737757 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f9fb5b-b2e3-43a5-a4e3-7fb7405c4bd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8842f728103446ba84c8efa9a9c7ae8, user_id=2b6d78cc0344441da955742b96cf91fd, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:18.740450 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f9fb5b-b2e3-43a5-a4e3-7fb7405c4bd5 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:18.753308 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1200/2380] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:12:18 2022] PUT /identity/v3/projects/d8842f728103446ba84c8efa9a9c7ae8/users/2b6d78cc0344441da955742b96cf91fd/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:12:18.763975 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6f310-0dca-4b75-9c5b-43600315d44d None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:18.768882 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6f310-0dca-4b75-9c5b-43600315d44d None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:18.769196 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6f310-0dca-4b75-9c5b-43600315d44d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:18.798775 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-47b6f310-0dca-4b75-9c5b-43600315d44d None None] MFA Rules not processed for user `2b6d78cc0344441da955742b96cf91fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:12:18.800010 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-47b6f310-0dca-4b75-9c5b-43600315d44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:18.865821 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1181/2381] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:12:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:12:18.870342 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f9fb5b-b2e3-43a5-a4e3-7fb7405c4bd5 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:18.932554 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f9fb5b-b2e3-43a5-a4e3-7fb7405c4bd5 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:18.937392 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fbed74-b4fd-4eea-befc-7fef464a3149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:18.937945 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fbed74-b4fd-4eea-befc-7fef464a3149 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:18.938229 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fbed74-b4fd-4eea-befc-7fef464a3149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:18.938508 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fbed74-b4fd-4eea-befc-7fef464a3149 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:18.939199 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fbed74-b4fd-4eea-befc-7fef464a3149 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:18.941132 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fbed74-b4fd-4eea-befc-7fef464a3149 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:18.966493 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1201/2382] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:12:18 2022] POST /identity/v3/projects => generated 387 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:12:18.971202 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b6f310-0dca-4b75-9c5b-43600315d44d None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:18.981789 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b6f310-0dca-4b75-9c5b-43600315d44d None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:19.039388 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b8c3e-da01-4ca4-ae93-ff9edc245b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:19.040080 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b8c3e-da01-4ca4-ae93-ff9edc245b29 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:19.040361 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b8c3e-da01-4ca4-ae93-ff9edc245b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:19.040637 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b8c3e-da01-4ca4-ae93-ff9edc245b29 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:19.041360 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475b8c3e-da01-4ca4-ae93-ff9edc245b29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:19.043292 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475b8c3e-da01-4ca4-ae93-ff9edc245b29 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:19.063559 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1182/2383] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:12:18 2022] POST /identity/v3/users => generated 300 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:12:19.067541 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fbed74-b4fd-4eea-befc-7fef464a3149 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:19.078477 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fbed74-b4fd-4eea-befc-7fef464a3149 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:19.083148 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ac08c-a54d-45af-a842-0a0d84eef469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:19.083840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ac08c-a54d-45af-a842-0a0d84eef469 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:19.084121 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ac08c-a54d-45af-a842-0a0d84eef469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:19.084395 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ac08c-a54d-45af-a842-0a0d84eef469 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:19.085122 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ac08c-a54d-45af-a842-0a0d84eef469 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:19.086707 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ac08c-a54d-45af-a842-0a0d84eef469 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:19.097628 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1202/2384] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:12:19.101604 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b8c3e-da01-4ca4-ae93-ff9edc245b29 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:19.111728 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b8c3e-da01-4ca4-ae93-ff9edc245b29 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:19.116285 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9066bdd-9be1-4306-9cbf-9c8582ffa6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:19.117148 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9066bdd-9be1-4306-9cbf-9c8582ffa6a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:19.117428 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9066bdd-9be1-4306-9cbf-9c8582ffa6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:19.117699 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9066bdd-9be1-4306-9cbf-9c8582ffa6a0 admin admin] PATH_INFO: `/v3/projects/a963f247b8b2407787f292b7871ed434/users/a424edf09c024f45b57a36e0da5f777a/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:19.132108 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9066bdd-9be1-4306-9cbf-9c8582ffa6a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a963f247b8b2407787f292b7871ed434, user_id=a424edf09c024f45b57a36e0da5f777a, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:19.134723 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9066bdd-9be1-4306-9cbf-9c8582ffa6a0 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:19.147857 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1183/2385] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:12:19 2022] PUT /identity/v3/projects/a963f247b8b2407787f292b7871ed434/users/a424edf09c024f45b57a36e0da5f777a/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:12:19.151672 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ac08c-a54d-45af-a842-0a0d84eef469 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:19.162120 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ac08c-a54d-45af-a842-0a0d84eef469 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:19.181680 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:12:19.220805 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25d336d-24dc-4639-a5f1-72bdd94ed3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:19.221469 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25d336d-24dc-4639-a5f1-72bdd94ed3f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:19.221750 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25d336d-24dc-4639-a5f1-72bdd94ed3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:19.222020 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25d336d-24dc-4639-a5f1-72bdd94ed3f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:19.222738 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25d336d-24dc-4639-a5f1-72bdd94ed3f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:19.224356 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25d336d-24dc-4639-a5f1-72bdd94ed3f4 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:19.234359 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1203/2386] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:12:19.238523 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9066bdd-9be1-4306-9cbf-9c8582ffa6a0 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:19.244032 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:12:19.251080 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9066bdd-9be1-4306-9cbf-9c8582ffa6a0 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:19.256104 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366c9d7a-c245-4539-a429-a0147b6236e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:19.256984 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c9d7a-c245-4539-a429-a0147b6236e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:19.257267 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c9d7a-c245-4539-a429-a0147b6236e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:19.257549 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c9d7a-c245-4539-a429-a0147b6236e9 admin admin] PATH_INFO: `/v3/projects/a963f247b8b2407787f292b7871ed434/users/a424edf09c024f45b57a36e0da5f777a/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:19.259538 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366c9d7a-c245-4539-a429-a0147b6236e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a963f247b8b2407787f292b7871ed434, user_id=a424edf09c024f45b57a36e0da5f777a, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:19.262153 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366c9d7a-c245-4539-a429-a0147b6236e9 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:19.273076 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1184/2387] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:12:19 2022] PUT /identity/v3/projects/a963f247b8b2407787f292b7871ed434/users/a424edf09c024f45b57a36e0da5f777a/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:12:19.284415 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b9dc4a-da51-49c4-addd-3c38c3815734 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:19.284705 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b9dc4a-da51-49c4-addd-3c38c3815734 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:19.285006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b9dc4a-da51-49c4-addd-3c38c3815734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:19.310648 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-b7b9dc4a-da51-49c4-addd-3c38c3815734 None None] MFA Rules not processed for user `a424edf09c024f45b57a36e0da5f777a`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:12:19.312335 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-b7b9dc4a-da51-49c4-addd-3c38c3815734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:19.381546 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1204/2388] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:12:19 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:12:19.393528 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366c9d7a-c245-4539-a429-a0147b6236e9 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:19.395246 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-366c9d7a-c245-4539-a429-a0147b6236e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:19.451785 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366c9d7a-c245-4539-a429-a0147b6236e9 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:19.456126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401fcf3c-6ef1-455f-a413-1f320fdce4fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:19.456765 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401fcf3c-6ef1-455f-a413-1f320fdce4fc service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:19.457051 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401fcf3c-6ef1-455f-a413-1f320fdce4fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:19.457330 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401fcf3c-6ef1-455f-a413-1f320fdce4fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:19.466572 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401fcf3c-6ef1-455f-a413-1f320fdce4fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:19.468092 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401fcf3c-6ef1-455f-a413-1f320fdce4fc service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:19.477727 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1185/2389] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:12:19 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:12:19.540535 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b9dc4a-da51-49c4-addd-3c38c3815734 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:19.606358 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b9dc4a-da51-49c4-addd-3c38c3815734 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:19.612506 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc4b80-2594-478d-9771-5879eefe8a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:19.613394 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc4b80-2594-478d-9771-5879eefe8a2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:19.613680 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc4b80-2594-478d-9771-5879eefe8a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:19.613953 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc4b80-2594-478d-9771-5879eefe8a2b admin admin] PATH_INFO: `/v3/users/2b6d78cc0344441da955742b96cf91fd` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:19.615162 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedc4b80-2594-478d-9771-5879eefe8a2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b6d78cc0344441da955742b96cf91fd)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:19.617103 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedc4b80-2594-478d-9771-5879eefe8a2b admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:19.667714 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-eedc4b80-2594-478d-9771-5879eefe8a2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b6d78cc0344441da955742b96cf91fd'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:19.668912 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:12:19.676776 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-eedc4b80-2594-478d-9771-5879eefe8a2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b6d78cc0344441da955742b96cf91fd'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:19.689369 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-eedc4b80-2594-478d-9771-5879eefe8a2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b6d78cc0344441da955742b96cf91fd'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:19.690076 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-eedc4b80-2594-478d-9771-5879eefe8a2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b6d78cc0344441da955742b96cf91fd'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:19.690676 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-eedc4b80-2594-478d-9771-5879eefe8a2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b6d78cc0344441da955742b96cf91fd'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:19.704641 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1205/2390] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:12:19 2022] DELETE /identity/v3/users/2b6d78cc0344441da955742b96cf91fd => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:12:19.709222 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947c8e0-f222-4229-97e0-afda01abdadb None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:19.709538 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947c8e0-f222-4229-97e0-afda01abdadb None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:19.709847 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947c8e0-f222-4229-97e0-afda01abdadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:19.728405 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-3947c8e0-f222-4229-97e0-afda01abdadb None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:12:19.729583 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-3947c8e0-f222-4229-97e0-afda01abdadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:19.788983 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1186/2391] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:12:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:12:19.799125 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc4b80-2594-478d-9771-5879eefe8a2b admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:19.800688 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-eedc4b80-2594-478d-9771-5879eefe8a2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:19.864355 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc4b80-2594-478d-9771-5879eefe8a2b admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:19.868089 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3c4536-358c-462a-963c-4acd197d9821 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:19.868742 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c4536-358c-462a-963c-4acd197d9821 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:19.869043 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c4536-358c-462a-963c-4acd197d9821 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:19.869318 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c4536-358c-462a-963c-4acd197d9821 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:19.879357 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3c4536-358c-462a-963c-4acd197d9821 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:19.880783 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3c4536-358c-462a-963c-4acd197d9821 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:19.890194 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1206/2392] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:12:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:12:20.161252 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947c8e0-f222-4229-97e0-afda01abdadb None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:20.162713 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-3947c8e0-f222-4229-97e0-afda01abdadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:20.175489 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947c8e0-f222-4229-97e0-afda01abdadb None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:20.180041 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c9aace-8f5b-4b9b-8aff-08ed8eafdcd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:20.180955 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c9aace-8f5b-4b9b-8aff-08ed8eafdcd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:20.181241 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c9aace-8f5b-4b9b-8aff-08ed8eafdcd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:20.181517 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c9aace-8f5b-4b9b-8aff-08ed8eafdcd4 admin admin] PATH_INFO: `/v3/projects/d8842f728103446ba84c8efa9a9c7ae8` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:20.182721 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c9aace-8f5b-4b9b-8aff-08ed8eafdcd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8842f728103446ba84c8efa9a9c7ae8)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:20.184690 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c9aace-8f5b-4b9b-8aff-08ed8eafdcd4 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:20.235803 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-b0c9aace-8f5b-4b9b-8aff-08ed8eafdcd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8842f728103446ba84c8efa9a9c7ae8', 'request_id': 'req-b0c9aace-8f5b-4b9b-8aff-08ed8eafdcd4'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:20.243057 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-b0c9aace-8f5b-4b9b-8aff-08ed8eafdcd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8842f728103446ba84c8efa9a9c7ae8', 'request_id': 'req-b0c9aace-8f5b-4b9b-8aff-08ed8eafdcd4'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:20.250321 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-b0c9aace-8f5b-4b9b-8aff-08ed8eafdcd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8842f728103446ba84c8efa9a9c7ae8', 'request_id': 'req-b0c9aace-8f5b-4b9b-8aff-08ed8eafdcd4'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:20.261078 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-b0c9aace-8f5b-4b9b-8aff-08ed8eafdcd4 admin admin] The token cache is being invalidate because project d8842f728103446ba84c8efa9a9c7ae8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:12:20.261366 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-b0c9aace-8f5b-4b9b-8aff-08ed8eafdcd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:20.262022 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-b0c9aace-8f5b-4b9b-8aff-08ed8eafdcd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:20.263587 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1187/2393] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:12:20 2022] DELETE /identity/v3/projects/d8842f728103446ba84c8efa9a9c7ae8 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:12:20.267868 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3c4536-358c-462a-963c-4acd197d9821 service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:20.277334 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3c4536-358c-462a-963c-4acd197d9821 service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:20.333833 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab24cf26-5d8d-41c8-a18c-9975364e6c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:20.334628 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab24cf26-5d8d-41c8-a18c-9975364e6c9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:20.334908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab24cf26-5d8d-41c8-a18c-9975364e6c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:20.335181 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab24cf26-5d8d-41c8-a18c-9975364e6c9a admin admin] PATH_INFO: `/v3/users/a424edf09c024f45b57a36e0da5f777a` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:20.336389 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab24cf26-5d8d-41c8-a18c-9975364e6c9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a424edf09c024f45b57a36e0da5f777a)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:20.338320 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab24cf26-5d8d-41c8-a18c-9975364e6c9a admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:20.384080 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-ab24cf26-5d8d-41c8-a18c-9975364e6c9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a424edf09c024f45b57a36e0da5f777a'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:20.391534 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-ab24cf26-5d8d-41c8-a18c-9975364e6c9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a424edf09c024f45b57a36e0da5f777a'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:20.401486 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:12:20.404025 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-ab24cf26-5d8d-41c8-a18c-9975364e6c9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a424edf09c024f45b57a36e0da5f777a'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:20.406244 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-ab24cf26-5d8d-41c8-a18c-9975364e6c9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a424edf09c024f45b57a36e0da5f777a'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:20.407335 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-ab24cf26-5d8d-41c8-a18c-9975364e6c9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a424edf09c024f45b57a36e0da5f777a'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:20.420913 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1207/2394] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:12:20 2022] DELETE /identity/v3/users/a424edf09c024f45b57a36e0da5f777a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:12:20.691693 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c9aace-8f5b-4b9b-8aff-08ed8eafdcd4 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:20.693221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-b0c9aace-8f5b-4b9b-8aff-08ed8eafdcd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:20.709618 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c9aace-8f5b-4b9b-8aff-08ed8eafdcd4 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:20.723807 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:12:20.774163 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b35014-9a23-47ac-bbab-01670c875625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:20.775721 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b35014-9a23-47ac-bbab-01670c875625 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:20.776006 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b35014-9a23-47ac-bbab-01670c875625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:20.776283 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b35014-9a23-47ac-bbab-01670c875625 admin admin] PATH_INFO: `/v3/projects/a963f247b8b2407787f292b7871ed434` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:20.777489 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b35014-9a23-47ac-bbab-01670c875625 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a963f247b8b2407787f292b7871ed434)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:20.779424 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b35014-9a23-47ac-bbab-01670c875625 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:20.833221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-24b35014-9a23-47ac-bbab-01670c875625 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a963f247b8b2407787f292b7871ed434', 'request_id': 'req-24b35014-9a23-47ac-bbab-01670c875625'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:20.842358 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-24b35014-9a23-47ac-bbab-01670c875625 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a963f247b8b2407787f292b7871ed434', 'request_id': 'req-24b35014-9a23-47ac-bbab-01670c875625'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:20.849821 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-24b35014-9a23-47ac-bbab-01670c875625 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a963f247b8b2407787f292b7871ed434', 'request_id': 'req-24b35014-9a23-47ac-bbab-01670c875625'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:20.864066 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-24b35014-9a23-47ac-bbab-01670c875625 admin admin] The token cache is being invalidate because project a963f247b8b2407787f292b7871ed434 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:12:20.864358 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-24b35014-9a23-47ac-bbab-01670c875625 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:20.865033 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-24b35014-9a23-47ac-bbab-01670c875625 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:20.866883 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1188/2395] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:12:20 2022] DELETE /identity/v3/projects/a963f247b8b2407787f292b7871ed434 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:12:20.871688 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a4f16-e399-4e28-a1d7-abde4d677333 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:20.872050 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a4f16-e399-4e28-a1d7-abde4d677333 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:20.872406 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a4f16-e399-4e28-a1d7-abde4d677333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:20.893315 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-2a6a4f16-e399-4e28-a1d7-abde4d677333 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:12:20.894621 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-2a6a4f16-e399-4e28-a1d7-abde4d677333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:20.958528 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1208/2396] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:12:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:12:20.966317 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c85e4-cae3-4314-8616-643f32e4a8db None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:20.966688 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c85e4-cae3-4314-8616-643f32e4a8db None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:20.967006 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c85e4-cae3-4314-8616-643f32e4a8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:20.985754 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-dd3c85e4-cae3-4314-8616-643f32e4a8db None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:12:20.986961 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-dd3c85e4-cae3-4314-8616-643f32e4a8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:20.994802 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1189/2397] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:12:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:12:20.998948 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6a4f16-e399-4e28-a1d7-abde4d677333 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:21.009123 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6a4f16-e399-4e28-a1d7-abde4d677333 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:21.014071 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb42b15-b0b9-40a9-8a27-9c6dd33c4b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:21.014710 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb42b15-b0b9-40a9-8a27-9c6dd33c4b45 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:21.015656 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb42b15-b0b9-40a9-8a27-9c6dd33c4b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:21.016032 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb42b15-b0b9-40a9-8a27-9c6dd33c4b45 admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:21.016760 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb42b15-b0b9-40a9-8a27-9c6dd33c4b45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:12:21.017075 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb42b15-b0b9-40a9-8a27-9c6dd33c4b45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:21.018663 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb42b15-b0b9-40a9-8a27-9c6dd33c4b45 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:21.030081 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1209/2398] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:12:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:12:21.034398 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3c85e4-cae3-4314-8616-643f32e4a8db None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:21.043722 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3c85e4-cae3-4314-8616-643f32e4a8db None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:21.048966 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc39c2df-5935-4ed6-8bd2-a57b5bdb83f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:21.049621 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39c2df-5935-4ed6-8bd2-a57b5bdb83f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:21.049911 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39c2df-5935-4ed6-8bd2-a57b5bdb83f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:21.050190 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39c2df-5935-4ed6-8bd2-a57b5bdb83f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:21.050895 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc39c2df-5935-4ed6-8bd2-a57b5bdb83f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:21.052882 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc39c2df-5935-4ed6-8bd2-a57b5bdb83f7 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:21.080432 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1190/2399] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:12:21 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:12:21.084473 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb42b15-b0b9-40a9-8a27-9c6dd33c4b45 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:21.097315 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb42b15-b0b9-40a9-8a27-9c6dd33c4b45 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:21.153362 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b4841-7712-4277-ae25-4d394086080f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:21.154034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b4841-7712-4277-ae25-4d394086080f admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:21.154320 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b4841-7712-4277-ae25-4d394086080f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:21.154600 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b4841-7712-4277-ae25-4d394086080f admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:21.155298 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3b4841-7712-4277-ae25-4d394086080f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:21.157478 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3b4841-7712-4277-ae25-4d394086080f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:21.180850 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1210/2400] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:12:21 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:12:21.182018 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc39c2df-5935-4ed6-8bd2-a57b5bdb83f7 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:21.193089 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc39c2df-5935-4ed6-8bd2-a57b5bdb83f7 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:21.197725 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf5c1b-667e-4cba-80b3-e92baf500b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:21.198383 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf5c1b-667e-4cba-80b3-e92baf500b8b admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:21.198670 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf5c1b-667e-4cba-80b3-e92baf500b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:21.198948 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf5c1b-667e-4cba-80b3-e92baf500b8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:21.199721 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cf5c1b-667e-4cba-80b3-e92baf500b8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:21.201314 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cf5c1b-667e-4cba-80b3-e92baf500b8b admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:21.212054 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1191/2401] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:12:21.215808 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b4841-7712-4277-ae25-4d394086080f admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:21.225602 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b4841-7712-4277-ae25-4d394086080f admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:21.230437 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15dc730-27b0-4f5d-84f9-57366b3edf34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:21.231086 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15dc730-27b0-4f5d-84f9-57366b3edf34 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:21.231407 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15dc730-27b0-4f5d-84f9-57366b3edf34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:21.231683 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15dc730-27b0-4f5d-84f9-57366b3edf34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:21.232406 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15dc730-27b0-4f5d-84f9-57366b3edf34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:21.234878 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15dc730-27b0-4f5d-84f9-57366b3edf34 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:21.245064 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1211/2402] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:12:21.249115 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf5c1b-667e-4cba-80b3-e92baf500b8b admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:21.260049 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf5c1b-667e-4cba-80b3-e92baf500b8b admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:21.264900 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c97932-b095-4868-aaed-24cfd9c92edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:21.265778 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c97932-b095-4868-aaed-24cfd9c92edf admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:21.266060 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c97932-b095-4868-aaed-24cfd9c92edf admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:21.266341 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c97932-b095-4868-aaed-24cfd9c92edf admin admin] PATH_INFO: `/v3/projects/27dd5773e0c84466932e61e5d8bb4cc5/users/b369a26ad3794619bd003fc73da7afdc/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:21.280961 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c97932-b095-4868-aaed-24cfd9c92edf admin admin] RBAC: Authorizing `identity:create_grant(project_id=27dd5773e0c84466932e61e5d8bb4cc5, user_id=b369a26ad3794619bd003fc73da7afdc, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:21.283634 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c97932-b095-4868-aaed-24cfd9c92edf admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:21.297035 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1192/2403] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:12:21 2022] PUT /identity/v3/projects/27dd5773e0c84466932e61e5d8bb4cc5/users/b369a26ad3794619bd003fc73da7afdc/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:12:21.307417 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a44ab6-1790-4b44-9c96-db604c68806b None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:21.307952 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a44ab6-1790-4b44-9c96-db604c68806b None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:21.308049 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a44ab6-1790-4b44-9c96-db604c68806b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:21.334905 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-c0a44ab6-1790-4b44-9c96-db604c68806b None None] MFA Rules not processed for user `b369a26ad3794619bd003fc73da7afdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:12:21.336178 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c0a44ab6-1790-4b44-9c96-db604c68806b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:21.378151 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:12:21.402874 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1212/2404] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:12:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:12:21.408156 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c97932-b095-4868-aaed-24cfd9c92edf admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:21.453218 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:12:21.472766 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c97932-b095-4868-aaed-24cfd9c92edf admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:21.477747 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83423ddf-fe35-4ec9-8e7c-5e2d273efbb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:21.478413 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83423ddf-fe35-4ec9-8e7c-5e2d273efbb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:21.478699 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83423ddf-fe35-4ec9-8e7c-5e2d273efbb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:21.478979 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83423ddf-fe35-4ec9-8e7c-5e2d273efbb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:21.479739 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83423ddf-fe35-4ec9-8e7c-5e2d273efbb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:21.481640 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83423ddf-fe35-4ec9-8e7c-5e2d273efbb5 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:21.512354 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1193/2405] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:12:21 2022] POST /identity/v3/projects => generated 379 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:12:21.513700 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a44ab6-1790-4b44-9c96-db604c68806b None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:21.524896 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a44ab6-1790-4b44-9c96-db604c68806b None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:21.587137 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fd049c-4520-4476-b990-07ef33d1b02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:21.587865 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd049c-4520-4476-b990-07ef33d1b02b admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:21.588151 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd049c-4520-4476-b990-07ef33d1b02b admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:21.588429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd049c-4520-4476-b990-07ef33d1b02b admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:21.589119 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fd049c-4520-4476-b990-07ef33d1b02b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:21.591036 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fd049c-4520-4476-b990-07ef33d1b02b admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:21.614465 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1213/2406] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:12:21 2022] POST /identity/v3/users => generated 296 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:12:21.618465 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83423ddf-fe35-4ec9-8e7c-5e2d273efbb5 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:21.628596 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83423ddf-fe35-4ec9-8e7c-5e2d273efbb5 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:21.633072 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890e85d0-4a8b-45e1-98c8-e06268544961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:21.633720 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e85d0-4a8b-45e1-98c8-e06268544961 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:21.634008 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e85d0-4a8b-45e1-98c8-e06268544961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:21.634281 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e85d0-4a8b-45e1-98c8-e06268544961 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:21.635035 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890e85d0-4a8b-45e1-98c8-e06268544961 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:21.636650 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890e85d0-4a8b-45e1-98c8-e06268544961 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:21.647010 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1194/2407] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:12:21.650937 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fd049c-4520-4476-b990-07ef33d1b02b admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:21.661362 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fd049c-4520-4476-b990-07ef33d1b02b admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:21.665975 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba3bb9c-2851-4c08-98ec-b38e2f450f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:21.666847 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba3bb9c-2851-4c08-98ec-b38e2f450f4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:21.667175 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba3bb9c-2851-4c08-98ec-b38e2f450f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:21.667499 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba3bb9c-2851-4c08-98ec-b38e2f450f4f admin admin] PATH_INFO: `/v3/projects/e021d87592e845cd92badec527a4328d/users/2eb7eefe9e6c46f1b13f94f5168f5f0b/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:21.682885 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba3bb9c-2851-4c08-98ec-b38e2f450f4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e021d87592e845cd92badec527a4328d, user_id=2eb7eefe9e6c46f1b13f94f5168f5f0b, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:21.685631 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba3bb9c-2851-4c08-98ec-b38e2f450f4f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:21.699499 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1214/2408] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:12:21 2022] PUT /identity/v3/projects/e021d87592e845cd92badec527a4328d/users/2eb7eefe9e6c46f1b13f94f5168f5f0b/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:12:21.703591 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890e85d0-4a8b-45e1-98c8-e06268544961 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:21.713654 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890e85d0-4a8b-45e1-98c8-e06268544961 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:21.772638 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec8e588-1843-4f29-b66e-afff8e04339b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:21.773328 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8e588-1843-4f29-b66e-afff8e04339b admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:21.773618 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8e588-1843-4f29-b66e-afff8e04339b admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:21.773901 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8e588-1843-4f29-b66e-afff8e04339b admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:21.774633 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec8e588-1843-4f29-b66e-afff8e04339b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:21.776280 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec8e588-1843-4f29-b66e-afff8e04339b admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:21.786934 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1195/2409] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 22:12:21.791308 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba3bb9c-2851-4c08-98ec-b38e2f450f4f admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:21.802243 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba3bb9c-2851-4c08-98ec-b38e2f450f4f admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:21.806822 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f989f86e-0210-4fcd-adbd-def4bbdc1f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:21.807755 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989f86e-0210-4fcd-adbd-def4bbdc1f04 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:21.808038 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989f86e-0210-4fcd-adbd-def4bbdc1f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:21.808313 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989f86e-0210-4fcd-adbd-def4bbdc1f04 admin admin] PATH_INFO: `/v3/projects/e021d87592e845cd92badec527a4328d/users/2eb7eefe9e6c46f1b13f94f5168f5f0b/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:21.810275 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f989f86e-0210-4fcd-adbd-def4bbdc1f04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e021d87592e845cd92badec527a4328d, user_id=2eb7eefe9e6c46f1b13f94f5168f5f0b, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:21.813048 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f989f86e-0210-4fcd-adbd-def4bbdc1f04 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:21.823151 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1215/2410] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:12:21 2022] PUT /identity/v3/projects/e021d87592e845cd92badec527a4328d/users/2eb7eefe9e6c46f1b13f94f5168f5f0b/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:12:21.835343 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b0246-d509-4c36-9bad-292ea4d06f55 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:21.835662 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b0246-d509-4c36-9bad-292ea4d06f55 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:21.835941 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b0246-d509-4c36-9bad-292ea4d06f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:21.868366 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-171b0246-d509-4c36-9bad-292ea4d06f55 None None] MFA Rules not processed for user `2eb7eefe9e6c46f1b13f94f5168f5f0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:12:21.869582 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-171b0246-d509-4c36-9bad-292ea4d06f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:21.943302 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1196/2411] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:12:21 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:12:21.955491 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f989f86e-0210-4fcd-adbd-def4bbdc1f04 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:21.957480 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-f989f86e-0210-4fcd-adbd-def4bbdc1f04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:22.021511 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f989f86e-0210-4fcd-adbd-def4bbdc1f04 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:22.025208 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44a62f6-23cb-4d65-9209-3e9b934b9514 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:22.025859 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a62f6-23cb-4d65-9209-3e9b934b9514 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:22.026145 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a62f6-23cb-4d65-9209-3e9b934b9514 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:22.026451 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a62f6-23cb-4d65-9209-3e9b934b9514 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:22.036875 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44a62f6-23cb-4d65-9209-3e9b934b9514 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:22.038321 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44a62f6-23cb-4d65-9209-3e9b934b9514 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:22.054156 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1216/2412] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:12:21 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:12:22.373947 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171b0246-d509-4c36-9bad-292ea4d06f55 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:22.385464 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171b0246-d509-4c36-9bad-292ea4d06f55 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:22.390885 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db6f50-6aab-4902-851f-f0bd2f1c4bef tempest-VolumeQuotaClassesTest-302704994 tempest-VolumeQuotaClassesTest-302704994-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eb7eefe9e6c46f1b13f94f5168f5f0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e021d87592e845cd92badec527a4328d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:22.391585 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db6f50-6aab-4902-851f-f0bd2f1c4bef tempest-VolumeQuotaClassesTest-302704994 tempest-VolumeQuotaClassesTest-302704994-project-admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:22.391871 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db6f50-6aab-4902-851f-f0bd2f1c4bef tempest-VolumeQuotaClassesTest-302704994 tempest-VolumeQuotaClassesTest-302704994-project-admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:22.392153 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db6f50-6aab-4902-851f-f0bd2f1c4bef tempest-VolumeQuotaClassesTest-302704994 tempest-VolumeQuotaClassesTest-302704994-project-admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:22.392888 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7db6f50-6aab-4902-851f-f0bd2f1c4bef tempest-VolumeQuotaClassesTest-302704994 tempest-VolumeQuotaClassesTest-302704994-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:12:22.393209 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7db6f50-6aab-4902-851f-f0bd2f1c4bef tempest-VolumeQuotaClassesTest-302704994 tempest-VolumeQuotaClassesTest-302704994-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:22.394823 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7db6f50-6aab-4902-851f-f0bd2f1c4bef tempest-VolumeQuotaClassesTest-302704994 tempest-VolumeQuotaClassesTest-302704994-project-admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:22.407022 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1197/2413] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:12:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:12:22.411259 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44a62f6-23cb-4d65-9209-3e9b934b9514 service cinder] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:22.420327 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44a62f6-23cb-4d65-9209-3e9b934b9514 service cinder] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:22.425810 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b43012-136b-4645-b596-4280388fc41e tempest-VolumeQuotaClassesTest-302704994 tempest-VolumeQuotaClassesTest-302704994-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eb7eefe9e6c46f1b13f94f5168f5f0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e021d87592e845cd92badec527a4328d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:22.426594 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b43012-136b-4645-b596-4280388fc41e tempest-VolumeQuotaClassesTest-302704994 tempest-VolumeQuotaClassesTest-302704994-project-admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:22.426892 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b43012-136b-4645-b596-4280388fc41e tempest-VolumeQuotaClassesTest-302704994 tempest-VolumeQuotaClassesTest-302704994-project-admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:22.427173 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b43012-136b-4645-b596-4280388fc41e tempest-VolumeQuotaClassesTest-302704994 tempest-VolumeQuotaClassesTest-302704994-project-admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:22.427894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b43012-136b-4645-b596-4280388fc41e tempest-VolumeQuotaClassesTest-302704994 tempest-VolumeQuotaClassesTest-302704994-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:22.429801 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b43012-136b-4645-b596-4280388fc41e tempest-VolumeQuotaClassesTest-302704994 tempest-VolumeQuotaClassesTest-302704994-project-admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:22.455478 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1217/2414] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:12:22 2022] POST /identity/v3/projects => generated 355 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:12:22.459770 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db6f50-6aab-4902-851f-f0bd2f1c4bef tempest-VolumeQuotaClassesTest-302704994 tempest-VolumeQuotaClassesTest-302704994-project-admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:22.469627 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db6f50-6aab-4902-851f-f0bd2f1c4bef tempest-VolumeQuotaClassesTest-302704994 tempest-VolumeQuotaClassesTest-302704994-project-admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:22.489530 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:12:22.526418 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adceed9-a65a-4e84-9e6e-e28a6580fd5d tempest-VolumeQuotaClassesTest-302704994 tempest-VolumeQuotaClassesTest-302704994-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eb7eefe9e6c46f1b13f94f5168f5f0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e021d87592e845cd92badec527a4328d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:22.527079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adceed9-a65a-4e84-9e6e-e28a6580fd5d tempest-VolumeQuotaClassesTest-302704994 tempest-VolumeQuotaClassesTest-302704994-project-admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:22.527390 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adceed9-a65a-4e84-9e6e-e28a6580fd5d tempest-VolumeQuotaClassesTest-302704994 tempest-VolumeQuotaClassesTest-302704994-project-admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:22.527670 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adceed9-a65a-4e84-9e6e-e28a6580fd5d tempest-VolumeQuotaClassesTest-302704994 tempest-VolumeQuotaClassesTest-302704994-project-admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:22.528387 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adceed9-a65a-4e84-9e6e-e28a6580fd5d tempest-VolumeQuotaClassesTest-302704994 tempest-VolumeQuotaClassesTest-302704994-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:12:22.528693 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adceed9-a65a-4e84-9e6e-e28a6580fd5d tempest-VolumeQuotaClassesTest-302704994 tempest-VolumeQuotaClassesTest-302704994-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:22.530269 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adceed9-a65a-4e84-9e6e-e28a6580fd5d tempest-VolumeQuotaClassesTest-302704994 tempest-VolumeQuotaClassesTest-302704994-project-admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:22.541798 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1198/2415] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:12:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:12:22.579622 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b43012-136b-4645-b596-4280388fc41e tempest-VolumeQuotaClassesTest-302704994 tempest-VolumeQuotaClassesTest-302704994-project-admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:22.589846 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b43012-136b-4645-b596-4280388fc41e tempest-VolumeQuotaClassesTest-302704994 tempest-VolumeQuotaClassesTest-302704994-project-admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:22.595451 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5f9325-f423-4b69-ad4d-8feb86b3d9fa tempest-VolumeQuotaClassesTest-302704994 tempest-VolumeQuotaClassesTest-302704994-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eb7eefe9e6c46f1b13f94f5168f5f0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e021d87592e845cd92badec527a4328d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:22.596584 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5f9325-f423-4b69-ad4d-8feb86b3d9fa tempest-VolumeQuotaClassesTest-302704994 tempest-VolumeQuotaClassesTest-302704994-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:22.596945 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5f9325-f423-4b69-ad4d-8feb86b3d9fa tempest-VolumeQuotaClassesTest-302704994 tempest-VolumeQuotaClassesTest-302704994-project-admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:22.597273 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5f9325-f423-4b69-ad4d-8feb86b3d9fa tempest-VolumeQuotaClassesTest-302704994 tempest-VolumeQuotaClassesTest-302704994-project-admin] PATH_INFO: `/v3/projects/f8a6f5d59db44ec6b5b4c0664c79c995` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:22.598751 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5f9325-f423-4b69-ad4d-8feb86b3d9fa tempest-VolumeQuotaClassesTest-302704994 tempest-VolumeQuotaClassesTest-302704994-project-admin] RBAC: Authorizing `identity:delete_project(project_id=f8a6f5d59db44ec6b5b4c0664c79c995)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:22.600713 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5f9325-f423-4b69-ad4d-8feb86b3d9fa tempest-VolumeQuotaClassesTest-302704994 tempest-VolumeQuotaClassesTest-302704994-project-admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:22.613992 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:12:22.649202 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-4f5f9325-f423-4b69-ad4d-8feb86b3d9fa tempest-VolumeQuotaClassesTest-302704994 tempest-VolumeQuotaClassesTest-302704994-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8a6f5d59db44ec6b5b4c0664c79c995', 'request_id': 'req-4f5f9325-f423-4b69-ad4d-8feb86b3d9fa'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:22.656045 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-4f5f9325-f423-4b69-ad4d-8feb86b3d9fa tempest-VolumeQuotaClassesTest-302704994 tempest-VolumeQuotaClassesTest-302704994-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8a6f5d59db44ec6b5b4c0664c79c995', 'request_id': 'req-4f5f9325-f423-4b69-ad4d-8feb86b3d9fa'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:22.660980 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-4f5f9325-f423-4b69-ad4d-8feb86b3d9fa tempest-VolumeQuotaClassesTest-302704994 tempest-VolumeQuotaClassesTest-302704994-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8a6f5d59db44ec6b5b4c0664c79c995', 'request_id': 'req-4f5f9325-f423-4b69-ad4d-8feb86b3d9fa'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:22.672664 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-4f5f9325-f423-4b69-ad4d-8feb86b3d9fa tempest-VolumeQuotaClassesTest-302704994 tempest-VolumeQuotaClassesTest-302704994-project-admin] The token cache is being invalidate because project f8a6f5d59db44ec6b5b4c0664c79c995 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:12:22.672888 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-4f5f9325-f423-4b69-ad4d-8feb86b3d9fa tempest-VolumeQuotaClassesTest-302704994 tempest-VolumeQuotaClassesTest-302704994-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:22.673524 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-4f5f9325-f423-4b69-ad4d-8feb86b3d9fa tempest-VolumeQuotaClassesTest-302704994 tempest-VolumeQuotaClassesTest-302704994-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:22.675035 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1218/2416] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:12:22 2022] DELETE /identity/v3/projects/f8a6f5d59db44ec6b5b4c0664c79c995 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:12:22.861683 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adceed9-a65a-4e84-9e6e-e28a6580fd5d tempest-VolumeQuotaClassesTest-302704994 tempest-VolumeQuotaClassesTest-302704994-project-admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:22.863507 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-3adceed9-a65a-4e84-9e6e-e28a6580fd5d tempest-VolumeQuotaClassesTest-302704994 tempest-VolumeQuotaClassesTest-302704994-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:22.938623 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adceed9-a65a-4e84-9e6e-e28a6580fd5d tempest-VolumeQuotaClassesTest-302704994 tempest-VolumeQuotaClassesTest-302704994-project-admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:22.998501 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e72e1a0-acee-4c75-9c18-fb186ad3911c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:22.999799 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e72e1a0-acee-4c75-9c18-fb186ad3911c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:23.000085 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e72e1a0-acee-4c75-9c18-fb186ad3911c admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:23.000360 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e72e1a0-acee-4c75-9c18-fb186ad3911c admin admin] PATH_INFO: `/v3/users/b369a26ad3794619bd003fc73da7afdc` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:23.001580 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e72e1a0-acee-4c75-9c18-fb186ad3911c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b369a26ad3794619bd003fc73da7afdc)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:23.003514 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e72e1a0-acee-4c75-9c18-fb186ad3911c admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:23.048618 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-3e72e1a0-acee-4c75-9c18-fb186ad3911c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b369a26ad3794619bd003fc73da7afdc'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:23.049270 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-3e72e1a0-acee-4c75-9c18-fb186ad3911c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b369a26ad3794619bd003fc73da7afdc'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:23.056663 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-3e72e1a0-acee-4c75-9c18-fb186ad3911c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b369a26ad3794619bd003fc73da7afdc'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:23.067414 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:12:23.069824 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-3e72e1a0-acee-4c75-9c18-fb186ad3911c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b369a26ad3794619bd003fc73da7afdc'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:23.079399 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-3e72e1a0-acee-4c75-9c18-fb186ad3911c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b369a26ad3794619bd003fc73da7afdc'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:23.085357 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1199/2417] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:12:22 2022] DELETE /identity/v3/users/b369a26ad3794619bd003fc73da7afdc => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:12:23.089794 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c9a5f1-7462-4105-bf07-4065151859cf None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:23.090067 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c9a5f1-7462-4105-bf07-4065151859cf None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:23.090349 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c9a5f1-7462-4105-bf07-4065151859cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:23.109812 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-95c9a5f1-7462-4105-bf07-4065151859cf None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:12:23.110867 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-95c9a5f1-7462-4105-bf07-4065151859cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:23.169251 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1219/2418] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:12:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:12:23.179723 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e72e1a0-acee-4c75-9c18-fb186ad3911c admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:23.181030 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-3e72e1a0-acee-4c75-9c18-fb186ad3911c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:23.238987 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e72e1a0-acee-4c75-9c18-fb186ad3911c admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:23.242898 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d4d013-939e-4409-bfe8-4c181429248c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:23.243433 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d4d013-939e-4409-bfe8-4c181429248c service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:23.243724 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d4d013-939e-4409-bfe8-4c181429248c service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:23.244002 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d4d013-939e-4409-bfe8-4c181429248c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:23.253917 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d4d013-939e-4409-bfe8-4c181429248c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:23.255439 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d4d013-939e-4409-bfe8-4c181429248c service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:23.265428 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1200/2419] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:12:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:12:23.561585 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c9a5f1-7462-4105-bf07-4065151859cf None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:23.563226 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-95c9a5f1-7462-4105-bf07-4065151859cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:23.575850 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c9a5f1-7462-4105-bf07-4065151859cf None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:23.581342 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bfeb5d-d508-4f65-b41e-cd25b2d4503e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:23.582239 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bfeb5d-d508-4f65-b41e-cd25b2d4503e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:23.582520 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bfeb5d-d508-4f65-b41e-cd25b2d4503e admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:23.582823 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bfeb5d-d508-4f65-b41e-cd25b2d4503e admin admin] PATH_INFO: `/v3/projects/27dd5773e0c84466932e61e5d8bb4cc5` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:23.583991 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bfeb5d-d508-4f65-b41e-cd25b2d4503e admin admin] RBAC: Authorizing `identity:delete_project(project_id=27dd5773e0c84466932e61e5d8bb4cc5)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:23.585923 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bfeb5d-d508-4f65-b41e-cd25b2d4503e admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:23.633052 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-05bfeb5d-d508-4f65-b41e-cd25b2d4503e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27dd5773e0c84466932e61e5d8bb4cc5', 'request_id': 'req-05bfeb5d-d508-4f65-b41e-cd25b2d4503e'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:23.640609 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-05bfeb5d-d508-4f65-b41e-cd25b2d4503e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27dd5773e0c84466932e61e5d8bb4cc5', 'request_id': 'req-05bfeb5d-d508-4f65-b41e-cd25b2d4503e'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:23.646205 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-05bfeb5d-d508-4f65-b41e-cd25b2d4503e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27dd5773e0c84466932e61e5d8bb4cc5', 'request_id': 'req-05bfeb5d-d508-4f65-b41e-cd25b2d4503e'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:23.657641 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-05bfeb5d-d508-4f65-b41e-cd25b2d4503e admin admin] The token cache is being invalidate because project 27dd5773e0c84466932e61e5d8bb4cc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:12:23.657923 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-05bfeb5d-d508-4f65-b41e-cd25b2d4503e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:23.658552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-05bfeb5d-d508-4f65-b41e-cd25b2d4503e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:23.660162 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1220/2420] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:12:23 2022] DELETE /identity/v3/projects/27dd5773e0c84466932e61e5d8bb4cc5 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:12:23.664080 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d4d013-939e-4409-bfe8-4c181429248c service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:23.673800 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d4d013-939e-4409-bfe8-4c181429248c service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:23.729447 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33ece06-92f2-4498-997e-57c1e6598a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:23.730232 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ece06-92f2-4498-997e-57c1e6598a48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:23.730510 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ece06-92f2-4498-997e-57c1e6598a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:23.730781 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ece06-92f2-4498-997e-57c1e6598a48 admin admin] PATH_INFO: `/v3/users/2eb7eefe9e6c46f1b13f94f5168f5f0b` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:23.731980 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33ece06-92f2-4498-997e-57c1e6598a48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2eb7eefe9e6c46f1b13f94f5168f5f0b)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:23.734552 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33ece06-92f2-4498-997e-57c1e6598a48 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:23.782493 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-b33ece06-92f2-4498-997e-57c1e6598a48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2eb7eefe9e6c46f1b13f94f5168f5f0b'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:23.783166 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-b33ece06-92f2-4498-997e-57c1e6598a48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2eb7eefe9e6c46f1b13f94f5168f5f0b'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:23.791173 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-b33ece06-92f2-4498-997e-57c1e6598a48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2eb7eefe9e6c46f1b13f94f5168f5f0b'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:23.805344 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-b33ece06-92f2-4498-997e-57c1e6598a48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2eb7eefe9e6c46f1b13f94f5168f5f0b'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:23.812583 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-b33ece06-92f2-4498-997e-57c1e6598a48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2eb7eefe9e6c46f1b13f94f5168f5f0b'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:23.818658 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1201/2421] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:12:23 2022] DELETE /identity/v3/users/2eb7eefe9e6c46f1b13f94f5168f5f0b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:12:24.088228 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bfeb5d-d508-4f65-b41e-cd25b2d4503e admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:24.090001 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-05bfeb5d-d508-4f65-b41e-cd25b2d4503e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:24.106132 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bfeb5d-d508-4f65-b41e-cd25b2d4503e admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:24.125478 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:12:24.167109 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41912570-b900-4f0c-b649-09f5a80670c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:24.168004 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41912570-b900-4f0c-b649-09f5a80670c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:24.168286 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41912570-b900-4f0c-b649-09f5a80670c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:24.168561 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41912570-b900-4f0c-b649-09f5a80670c3 admin admin] PATH_INFO: `/v3/projects/e021d87592e845cd92badec527a4328d` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:24.169736 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41912570-b900-4f0c-b649-09f5a80670c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e021d87592e845cd92badec527a4328d)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:24.171734 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41912570-b900-4f0c-b649-09f5a80670c3 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:24.216757 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-41912570-b900-4f0c-b649-09f5a80670c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e021d87592e845cd92badec527a4328d', 'request_id': 'req-41912570-b900-4f0c-b649-09f5a80670c3'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:24.224717 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-41912570-b900-4f0c-b649-09f5a80670c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e021d87592e845cd92badec527a4328d', 'request_id': 'req-41912570-b900-4f0c-b649-09f5a80670c3'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:24.230341 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-41912570-b900-4f0c-b649-09f5a80670c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e021d87592e845cd92badec527a4328d', 'request_id': 'req-41912570-b900-4f0c-b649-09f5a80670c3'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:24.241303 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-41912570-b900-4f0c-b649-09f5a80670c3 admin admin] The token cache is being invalidate because project e021d87592e845cd92badec527a4328d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:12:24.241585 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-41912570-b900-4f0c-b649-09f5a80670c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:24.242204 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-41912570-b900-4f0c-b649-09f5a80670c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:24.243734 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1221/2422] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:12:24 2022] DELETE /identity/v3/projects/e021d87592e845cd92badec527a4328d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:12:24.248593 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082d774-93da-4c0f-9f41-865367a14fe2 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:24.249209 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082d774-93da-4c0f-9f41-865367a14fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:24.249209 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082d774-93da-4c0f-9f41-865367a14fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:24.261936 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:12:24.271293 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-2082d774-93da-4c0f-9f41-865367a14fe2 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:12:24.273204 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-2082d774-93da-4c0f-9f41-865367a14fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:24.334634 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1202/2423] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:12:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:12:24.342542 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3f67d-d619-4dcd-a213-a068020b0285 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:24.342845 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3f67d-d619-4dcd-a213-a068020b0285 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:24.343745 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3f67d-d619-4dcd-a213-a068020b0285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:24.362531 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-7ed3f67d-d619-4dcd-a213-a068020b0285 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:12:24.363787 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-7ed3f67d-d619-4dcd-a213-a068020b0285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:24.372166 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1222/2424] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:12:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:12:24.376786 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2082d774-93da-4c0f-9f41-865367a14fe2 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:24.386677 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2082d774-93da-4c0f-9f41-865367a14fe2 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:24.392750 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f29c64-68e7-4497-a7b6-0b7ff61ecd25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:24.393368 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f29c64-68e7-4497-a7b6-0b7ff61ecd25 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:24.393660 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f29c64-68e7-4497-a7b6-0b7ff61ecd25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:24.393925 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f29c64-68e7-4497-a7b6-0b7ff61ecd25 admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:24.394657 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f29c64-68e7-4497-a7b6-0b7ff61ecd25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:12:24.394960 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f29c64-68e7-4497-a7b6-0b7ff61ecd25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:24.396639 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f29c64-68e7-4497-a7b6-0b7ff61ecd25 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:24.410083 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1203/2425] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:12:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:12:24.414514 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed3f67d-d619-4dcd-a213-a068020b0285 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:24.424160 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed3f67d-d619-4dcd-a213-a068020b0285 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:24.430103 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b455a41-403d-42af-88fc-e52e940906e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:24.431234 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b455a41-403d-42af-88fc-e52e940906e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:24.431567 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b455a41-403d-42af-88fc-e52e940906e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:24.431921 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b455a41-403d-42af-88fc-e52e940906e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:24.432633 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b455a41-403d-42af-88fc-e52e940906e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:24.434550 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b455a41-403d-42af-88fc-e52e940906e2 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:24.465123 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1223/2426] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:12:24 2022] POST /identity/v3/projects => generated 361 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:12:24.469537 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f29c64-68e7-4497-a7b6-0b7ff61ecd25 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:24.479821 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f29c64-68e7-4497-a7b6-0b7ff61ecd25 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:24.541957 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6b1940-c48c-48d8-a17c-bacce6d1d1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:24.542633 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b1940-c48c-48d8-a17c-bacce6d1d1fb admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:24.542912 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b1940-c48c-48d8-a17c-bacce6d1d1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:24.543182 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b1940-c48c-48d8-a17c-bacce6d1d1fb admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:24.543930 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6b1940-c48c-48d8-a17c-bacce6d1d1fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:24.545849 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6b1940-c48c-48d8-a17c-bacce6d1d1fb admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:24.572837 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1204/2427] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:12:24 2022] POST /identity/v3/users => generated 281 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:12:24.574085 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b455a41-403d-42af-88fc-e52e940906e2 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:24.585288 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b455a41-403d-42af-88fc-e52e940906e2 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:24.589946 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2e6de-9eb0-4591-a18d-88b361f4a81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:24.590616 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2e6de-9eb0-4591-a18d-88b361f4a81b admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:24.590902 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2e6de-9eb0-4591-a18d-88b361f4a81b admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:24.591179 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2e6de-9eb0-4591-a18d-88b361f4a81b admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:24.592003 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de2e6de-9eb0-4591-a18d-88b361f4a81b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:24.593889 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de2e6de-9eb0-4591-a18d-88b361f4a81b admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:24.606472 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1224/2428] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:12:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:12:24.610566 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6b1940-c48c-48d8-a17c-bacce6d1d1fb admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:24.620469 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6b1940-c48c-48d8-a17c-bacce6d1d1fb admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:24.625823 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a7e4ae-d756-4cf9-b8b7-8e78bb553ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:24.626480 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7e4ae-d756-4cf9-b8b7-8e78bb553ced admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:24.626764 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7e4ae-d756-4cf9-b8b7-8e78bb553ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:24.627067 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7e4ae-d756-4cf9-b8b7-8e78bb553ced admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:24.627856 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a7e4ae-d756-4cf9-b8b7-8e78bb553ced admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:24.629429 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a7e4ae-d756-4cf9-b8b7-8e78bb553ced admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:24.639390 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1205/2429] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:12:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:12:24.644472 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2e6de-9eb0-4591-a18d-88b361f4a81b admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:24.655166 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2e6de-9eb0-4591-a18d-88b361f4a81b admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:24.660115 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a4fd64-0ca6-48d5-be8e-d55a61f84587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:24.660996 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4fd64-0ca6-48d5-be8e-d55a61f84587 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:24.661284 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4fd64-0ca6-48d5-be8e-d55a61f84587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:24.661566 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4fd64-0ca6-48d5-be8e-d55a61f84587 admin admin] PATH_INFO: `/v3/projects/8a9edd43417b498f9d977b512de445f5/users/68006cbb92ff4718bde541e74e5adab8/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:24.677720 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a4fd64-0ca6-48d5-be8e-d55a61f84587 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a9edd43417b498f9d977b512de445f5, user_id=68006cbb92ff4718bde541e74e5adab8, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:24.680653 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a4fd64-0ca6-48d5-be8e-d55a61f84587 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:24.695413 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1225/2430] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:12:24 2022] PUT /identity/v3/projects/8a9edd43417b498f9d977b512de445f5/users/68006cbb92ff4718bde541e74e5adab8/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:12:24.710912 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce1489-9702-483a-9e31-be15fdab75f4 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:24.711203 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce1489-9702-483a-9e31-be15fdab75f4 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:24.711510 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce1489-9702-483a-9e31-be15fdab75f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:24.738712 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-07ce1489-9702-483a-9e31-be15fdab75f4 None None] MFA Rules not processed for user `68006cbb92ff4718bde541e74e5adab8`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:12:24.739972 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-07ce1489-9702-483a-9e31-be15fdab75f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:24.814305 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1206/2431] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:12:24 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:12:24.840406 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a4fd64-0ca6-48d5-be8e-d55a61f84587 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:24.842169 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-36a4fd64-0ca6-48d5-be8e-d55a61f84587 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:24.896575 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:12:24.900291 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a4fd64-0ca6-48d5-be8e-d55a61f84587 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:24.904746 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e7d34-1114-425c-aace-d9b44b351441 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:24.905515 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e7d34-1114-425c-aace-d9b44b351441 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:24.905851 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e7d34-1114-425c-aace-d9b44b351441 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:24.906339 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e7d34-1114-425c-aace-d9b44b351441 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:24.914732 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354e7d34-1114-425c-aace-d9b44b351441 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:24.916180 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354e7d34-1114-425c-aace-d9b44b351441 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:24.925489 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1226/2432] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:12:24 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:12:24.939863 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce1489-9702-483a-9e31-be15fdab75f4 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:24.944052 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:12:25.007350 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce1489-9702-483a-9e31-be15fdab75f4 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:25.011916 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4670946-7d5a-49d4-92a9-dc6895711047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:25.012789 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4670946-7d5a-49d4-92a9-dc6895711047 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:25.013076 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4670946-7d5a-49d4-92a9-dc6895711047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:25.013351 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4670946-7d5a-49d4-92a9-dc6895711047 admin admin] PATH_INFO: `/v3/users/68006cbb92ff4718bde541e74e5adab8` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:25.014539 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4670946-7d5a-49d4-92a9-dc6895711047 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68006cbb92ff4718bde541e74e5adab8)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:25.016736 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4670946-7d5a-49d4-92a9-dc6895711047 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:25.063290 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d4670946-7d5a-49d4-92a9-dc6895711047 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68006cbb92ff4718bde541e74e5adab8'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:25.063897 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d4670946-7d5a-49d4-92a9-dc6895711047 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68006cbb92ff4718bde541e74e5adab8'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:25.071007 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d4670946-7d5a-49d4-92a9-dc6895711047 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68006cbb92ff4718bde541e74e5adab8'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:25.083439 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d4670946-7d5a-49d4-92a9-dc6895711047 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68006cbb92ff4718bde541e74e5adab8'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:25.090071 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d4670946-7d5a-49d4-92a9-dc6895711047 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68006cbb92ff4718bde541e74e5adab8'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:25.096415 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1207/2433] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:12:24 2022] DELETE /identity/v3/users/68006cbb92ff4718bde541e74e5adab8 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:12:25.100917 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ba196-3680-4d86-abae-451701411a24 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:25.101231 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ba196-3680-4d86-abae-451701411a24 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:25.101511 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ba196-3680-4d86-abae-451701411a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:25.119433 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-cd5ba196-3680-4d86-abae-451701411a24 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:12:25.120611 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-cd5ba196-3680-4d86-abae-451701411a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:25.180840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1227/2434] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:12:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:12:25.191210 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4670946-7d5a-49d4-92a9-dc6895711047 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:25.192686 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-d4670946-7d5a-49d4-92a9-dc6895711047 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:25.249502 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4670946-7d5a-49d4-92a9-dc6895711047 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:25.253189 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a8d9c-24a5-49f3-8596-996981965a6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:25.253834 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a8d9c-24a5-49f3-8596-996981965a6d service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:25.254121 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a8d9c-24a5-49f3-8596-996981965a6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:25.254401 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a8d9c-24a5-49f3-8596-996981965a6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:25.263380 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953a8d9c-24a5-49f3-8596-996981965a6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:25.264853 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953a8d9c-24a5-49f3-8596-996981965a6d service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:25.274512 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1208/2435] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:12:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:12:25.622438 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ba196-3680-4d86-abae-451701411a24 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:25.623990 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-cd5ba196-3680-4d86-abae-451701411a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:25.636479 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ba196-3680-4d86-abae-451701411a24 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:25.646851 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ce9364-d63e-45bf-9836-16bf19b9fe54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:25.648000 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce9364-d63e-45bf-9836-16bf19b9fe54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:25.648351 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce9364-d63e-45bf-9836-16bf19b9fe54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:25.648812 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce9364-d63e-45bf-9836-16bf19b9fe54 admin admin] PATH_INFO: `/v3/projects/8a9edd43417b498f9d977b512de445f5` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:25.650219 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ce9364-d63e-45bf-9836-16bf19b9fe54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a9edd43417b498f9d977b512de445f5)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:25.652474 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ce9364-d63e-45bf-9836-16bf19b9fe54 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:25.712184 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-b3ce9364-d63e-45bf-9836-16bf19b9fe54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a9edd43417b498f9d977b512de445f5', 'request_id': 'req-b3ce9364-d63e-45bf-9836-16bf19b9fe54'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:25.721051 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-b3ce9364-d63e-45bf-9836-16bf19b9fe54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a9edd43417b498f9d977b512de445f5', 'request_id': 'req-b3ce9364-d63e-45bf-9836-16bf19b9fe54'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:25.726904 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-b3ce9364-d63e-45bf-9836-16bf19b9fe54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a9edd43417b498f9d977b512de445f5', 'request_id': 'req-b3ce9364-d63e-45bf-9836-16bf19b9fe54'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:25.739833 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-b3ce9364-d63e-45bf-9836-16bf19b9fe54 admin admin] The token cache is being invalidate because project 8a9edd43417b498f9d977b512de445f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:12:25.740141 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-b3ce9364-d63e-45bf-9836-16bf19b9fe54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:25.740803 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-b3ce9364-d63e-45bf-9836-16bf19b9fe54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:12:25.742326 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1228/2436] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:12:25 2022] DELETE /identity/v3/projects/8a9edd43417b498f9d977b512de445f5 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:12:25.747245 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9dc5c-0fa3-4690-a361-bfe6e0862987 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:25.747640 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9dc5c-0fa3-4690-a361-bfe6e0862987 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:25.747942 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9dc5c-0fa3-4690-a361-bfe6e0862987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:25.769749 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-52e9dc5c-0fa3-4690-a361-bfe6e0862987 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:12:25.770699 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-52e9dc5c-0fa3-4690-a361-bfe6e0862987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:25.791854 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:12:25.841538 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1209/2437] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:12:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:12:25.908454 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633cf77-0be3-44aa-bc34-b1c6c15e1c22 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:25.908849 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633cf77-0be3-44aa-bc34-b1c6c15e1c22 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:25.909195 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633cf77-0be3-44aa-bc34-b1c6c15e1c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:25.928072 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-b633cf77-0be3-44aa-bc34-b1c6c15e1c22 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:12:25.929355 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-b633cf77-0be3-44aa-bc34-b1c6c15e1c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:25.938228 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1229/2438] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:12:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:12:25.942386 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9dc5c-0fa3-4690-a361-bfe6e0862987 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:25.952240 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9dc5c-0fa3-4690-a361-bfe6e0862987 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:25.957491 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e68fa64-9fd9-4718-aa29-2d4e923beb19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:25.958224 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e68fa64-9fd9-4718-aa29-2d4e923beb19 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:25.958575 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e68fa64-9fd9-4718-aa29-2d4e923beb19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:25.958918 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e68fa64-9fd9-4718-aa29-2d4e923beb19 admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:25.959770 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e68fa64-9fd9-4718-aa29-2d4e923beb19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:12:25.960142 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e68fa64-9fd9-4718-aa29-2d4e923beb19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:25.961800 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e68fa64-9fd9-4718-aa29-2d4e923beb19 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:25.973643 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1210/2439] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:12:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:12:25.978238 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b633cf77-0be3-44aa-bc34-b1c6c15e1c22 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:25.987695 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b633cf77-0be3-44aa-bc34-b1c6c15e1c22 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:25.992341 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad9c6dc-73ca-4079-9b6e-954912f33cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:25.992994 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9c6dc-73ca-4079-9b6e-954912f33cb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:25.993277 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9c6dc-73ca-4079-9b6e-954912f33cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:25.993553 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9c6dc-73ca-4079-9b6e-954912f33cb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:25.994245 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad9c6dc-73ca-4079-9b6e-954912f33cb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:25.996311 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad9c6dc-73ca-4079-9b6e-954912f33cb3 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:26.028093 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1230/2440] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:12:25 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:12:26.029551 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e68fa64-9fd9-4718-aa29-2d4e923beb19 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:26.039556 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e68fa64-9fd9-4718-aa29-2d4e923beb19 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:26.099447 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc8545c-979a-4815-a32c-3aef9f27d94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:26.100120 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8545c-979a-4815-a32c-3aef9f27d94a admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:26.100408 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8545c-979a-4815-a32c-3aef9f27d94a admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:26.100691 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8545c-979a-4815-a32c-3aef9f27d94a admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:26.101378 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc8545c-979a-4815-a32c-3aef9f27d94a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:26.103609 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc8545c-979a-4815-a32c-3aef9f27d94a admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:26.126191 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1211/2441] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:12:26 2022] POST /identity/v3/users => generated 283 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:12:26.127444 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad9c6dc-73ca-4079-9b6e-954912f33cb3 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:26.137903 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad9c6dc-73ca-4079-9b6e-954912f33cb3 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:26.143542 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc38b945-0fbb-4c1b-a1c0-d28bac16944f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:26.144203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38b945-0fbb-4c1b-a1c0-d28bac16944f admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:26.144513 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38b945-0fbb-4c1b-a1c0-d28bac16944f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:26.144790 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38b945-0fbb-4c1b-a1c0-d28bac16944f admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:26.145548 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc38b945-0fbb-4c1b-a1c0-d28bac16944f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:26.147167 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc38b945-0fbb-4c1b-a1c0-d28bac16944f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:26.160936 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1231/2442] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:12:26.165135 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc8545c-979a-4815-a32c-3aef9f27d94a admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:26.180303 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc8545c-979a-4815-a32c-3aef9f27d94a admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:26.184938 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7893bb-5140-4359-a942-53960762a78a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:26.185603 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7893bb-5140-4359-a942-53960762a78a admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:26.185888 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7893bb-5140-4359-a942-53960762a78a admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:26.194643 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7893bb-5140-4359-a942-53960762a78a admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:26.195520 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7893bb-5140-4359-a942-53960762a78a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:26.197144 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7893bb-5140-4359-a942-53960762a78a admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:26.214040 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1212/2443] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:12:26.218445 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc38b945-0fbb-4c1b-a1c0-d28bac16944f admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:26.228679 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc38b945-0fbb-4c1b-a1c0-d28bac16944f admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:26.233211 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5567039-1ec6-4a48-8a9a-d83d80134402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:26.234194 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5567039-1ec6-4a48-8a9a-d83d80134402 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:26.234472 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5567039-1ec6-4a48-8a9a-d83d80134402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:26.234743 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5567039-1ec6-4a48-8a9a-d83d80134402 admin admin] PATH_INFO: `/v3/projects/b1e0ded8215848599fe003e8dca921ff/users/49288f658bff4c72b9af8e5f60a55b98/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:26.248766 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5567039-1ec6-4a48-8a9a-d83d80134402 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1e0ded8215848599fe003e8dca921ff, user_id=49288f658bff4c72b9af8e5f60a55b98, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:26.251926 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5567039-1ec6-4a48-8a9a-d83d80134402 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:26.264168 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1232/2444] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:12:26 2022] PUT /identity/v3/projects/b1e0ded8215848599fe003e8dca921ff/users/49288f658bff4c72b9af8e5f60a55b98/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:12:26.274778 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86106bb-76ff-4dc4-a837-9c24c25850a3 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:26.275250 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86106bb-76ff-4dc4-a837-9c24c25850a3 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:26.275371 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86106bb-76ff-4dc4-a837-9c24c25850a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:26.302921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-a86106bb-76ff-4dc4-a837-9c24c25850a3 None None] MFA Rules not processed for user `49288f658bff4c72b9af8e5f60a55b98`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:12:26.304173 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-a86106bb-76ff-4dc4-a837-9c24c25850a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:26.369713 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1213/2445] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:12:26 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:12:26.382309 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5567039-1ec6-4a48-8a9a-d83d80134402 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:26.384038 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-e5567039-1ec6-4a48-8a9a-d83d80134402 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:26.397489 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:12:26.445520 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5567039-1ec6-4a48-8a9a-d83d80134402 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:26.449234 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b324f4d-aad3-43df-80ba-cce060ad19a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:26.449890 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b324f4d-aad3-43df-80ba-cce060ad19a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:26.450194 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b324f4d-aad3-43df-80ba-cce060ad19a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:26.450456 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b324f4d-aad3-43df-80ba-cce060ad19a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:26.459695 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b324f4d-aad3-43df-80ba-cce060ad19a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:26.461136 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b324f4d-aad3-43df-80ba-cce060ad19a3 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:26.470475 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1233/2446] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:12:26 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:12:31.473916 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bae56b-422a-4d57-be0b-a368dab68fdd None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:31.474431 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bae56b-422a-4d57-be0b-a368dab68fdd None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:31.474534 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bae56b-422a-4d57-be0b-a368dab68fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:31.493925 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-c6bae56b-422a-4d57-be0b-a368dab68fdd None None] MFA Rules not processed for user `607b70b4d6e14c95ae2880dd3577c8ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:12:31.495151 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-c6bae56b-422a-4d57-be0b-a368dab68fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:31.508090 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:12:31.553201 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1214/2447] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:12:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:12:31.873605 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b324f4d-aad3-43df-80ba-cce060ad19a3 service cinder] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:12:31.875719 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-5b324f4d-aad3-43df-80ba-cce060ad19a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:31.939010 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b324f4d-aad3-43df-80ba-cce060ad19a3 service cinder] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:12:31.943072 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32bba45-12ef-4e61-8356-b95dd4c00b77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:12:31.943843 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32bba45-12ef-4e61-8356-b95dd4c00b77 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:12:31.944198 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32bba45-12ef-4e61-8356-b95dd4c00b77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:12:31.944547 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32bba45-12ef-4e61-8356-b95dd4c00b77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:12:31.946647 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-b32bba45-12ef-4e61-8356-b95dd4c00b77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:12:32.017893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32bba45-12ef-4e61-8356-b95dd4c00b77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:12:32.019376 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32bba45-12ef-4e61-8356-b95dd4c00b77 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:12:32.029698 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1234/2448] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:12:31 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:12:41.546200 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=16 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:12:42.025003 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=71 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:13:15.470005 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bae56b-422a-4d57-be0b-a368dab68fdd None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:13:15.534292 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bae56b-422a-4d57-be0b-a368dab68fdd None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:13:15.539279 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69421f88-be0c-41ca-bdcd-8e0aa44297d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:13:15.540244 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69421f88-be0c-41ca-bdcd-8e0aa44297d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:13:15.540525 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69421f88-be0c-41ca-bdcd-8e0aa44297d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:13:15.540798 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69421f88-be0c-41ca-bdcd-8e0aa44297d6 admin admin] PATH_INFO: `/v3/users/49288f658bff4c72b9af8e5f60a55b98` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:13:15.541990 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69421f88-be0c-41ca-bdcd-8e0aa44297d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49288f658bff4c72b9af8e5f60a55b98)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:13:15.543931 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69421f88-be0c-41ca-bdcd-8e0aa44297d6 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:13:15.591741 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-69421f88-be0c-41ca-bdcd-8e0aa44297d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49288f658bff4c72b9af8e5f60a55b98'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:13:15.592399 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-69421f88-be0c-41ca-bdcd-8e0aa44297d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49288f658bff4c72b9af8e5f60a55b98'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:13:15.602318 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-69421f88-be0c-41ca-bdcd-8e0aa44297d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49288f658bff4c72b9af8e5f60a55b98'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:13:15.621376 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-69421f88-be0c-41ca-bdcd-8e0aa44297d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49288f658bff4c72b9af8e5f60a55b98'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:13:15.634551 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-69421f88-be0c-41ca-bdcd-8e0aa44297d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49288f658bff4c72b9af8e5f60a55b98'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:13:15.643884 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1215/2449] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:13:15 2022] DELETE /identity/v3/users/49288f658bff4c72b9af8e5f60a55b98 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:13:15.645681 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee3443-a58a-468d-b1ae-265708db7f99 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:13:15.646222 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee3443-a58a-468d-b1ae-265708db7f99 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:13:15.646325 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee3443-a58a-468d-b1ae-265708db7f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:13:15.663861 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-d6ee3443-a58a-468d-b1ae-265708db7f99 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:13:15.665059 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d6ee3443-a58a-468d-b1ae-265708db7f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:13:15.726096 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1235/2450] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:13:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:13:15.736102 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69421f88-be0c-41ca-bdcd-8e0aa44297d6 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:13:15.737582 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-69421f88-be0c-41ca-bdcd-8e0aa44297d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:13:15.796180 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69421f88-be0c-41ca-bdcd-8e0aa44297d6 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:13:15.799772 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0601b6f-b73d-4fdb-80ee-8effe23e7845 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:13:15.800414 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0601b6f-b73d-4fdb-80ee-8effe23e7845 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:13:15.800691 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0601b6f-b73d-4fdb-80ee-8effe23e7845 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:13:15.800964 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0601b6f-b73d-4fdb-80ee-8effe23e7845 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:13:15.810507 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0601b6f-b73d-4fdb-80ee-8effe23e7845 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:13:15.811969 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0601b6f-b73d-4fdb-80ee-8effe23e7845 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:13:15.821836 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1216/2451] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:13:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:13:16.086168 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee3443-a58a-468d-b1ae-265708db7f99 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:13:16.087655 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d6ee3443-a58a-468d-b1ae-265708db7f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:13:16.100495 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee3443-a58a-468d-b1ae-265708db7f99 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:13:16.105356 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd945e6a-88c2-4b77-9e56-ef2b1cad3fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:13:16.106246 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd945e6a-88c2-4b77-9e56-ef2b1cad3fe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:13:16.106530 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd945e6a-88c2-4b77-9e56-ef2b1cad3fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:13:16.106809 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd945e6a-88c2-4b77-9e56-ef2b1cad3fe5 admin admin] PATH_INFO: `/v3/projects/b1e0ded8215848599fe003e8dca921ff` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:13:16.108000 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd945e6a-88c2-4b77-9e56-ef2b1cad3fe5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1e0ded8215848599fe003e8dca921ff)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:13:16.109913 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd945e6a-88c2-4b77-9e56-ef2b1cad3fe5 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:13:16.157300 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-dd945e6a-88c2-4b77-9e56-ef2b1cad3fe5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1e0ded8215848599fe003e8dca921ff', 'request_id': 'req-dd945e6a-88c2-4b77-9e56-ef2b1cad3fe5'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:13:16.168632 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-dd945e6a-88c2-4b77-9e56-ef2b1cad3fe5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1e0ded8215848599fe003e8dca921ff', 'request_id': 'req-dd945e6a-88c2-4b77-9e56-ef2b1cad3fe5'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:13:16.174851 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-dd945e6a-88c2-4b77-9e56-ef2b1cad3fe5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1e0ded8215848599fe003e8dca921ff', 'request_id': 'req-dd945e6a-88c2-4b77-9e56-ef2b1cad3fe5'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:13:16.188317 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-dd945e6a-88c2-4b77-9e56-ef2b1cad3fe5 admin admin] The token cache is being invalidate because project b1e0ded8215848599fe003e8dca921ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:13:16.188744 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-dd945e6a-88c2-4b77-9e56-ef2b1cad3fe5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:13:16.189496 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-dd945e6a-88c2-4b77-9e56-ef2b1cad3fe5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:13:16.191114 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1236/2452] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:13:16 2022] DELETE /identity/v3/projects/b1e0ded8215848599fe003e8dca921ff => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:13:17.052565 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0601b6f-b73d-4fdb-80ee-8effe23e7845 service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:13:17.054011 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-e0601b6f-b73d-4fdb-80ee-8effe23e7845 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:13:17.081453 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:13:17.112970 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0601b6f-b73d-4fdb-80ee-8effe23e7845 service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:13:17.160381 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b4c908-1167-4322-b6ac-9d147947c8be service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96987b04471944bc86ab44ff9936c732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:13:17.161026 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4c908-1167-4322-b6ac-9d147947c8be service glance] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:13:17.161311 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4c908-1167-4322-b6ac-9d147947c8be service glance] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:13:17.161593 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4c908-1167-4322-b6ac-9d147947c8be service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:13:17.163434 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-03b4c908-1167-4322-b6ac-9d147947c8be service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:13:17.234709 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b4c908-1167-4322-b6ac-9d147947c8be service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:13:17.236160 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b4c908-1167-4322-b6ac-9d147947c8be service glance] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:13:17.247385 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1217/2453] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:13:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:13:17.484226 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd945e6a-88c2-4b77-9e56-ef2b1cad3fe5 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:13:17.485643 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-dd945e6a-88c2-4b77-9e56-ef2b1cad3fe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:13:17.542569 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd945e6a-88c2-4b77-9e56-ef2b1cad3fe5 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:13:17.546093 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121dfc86-8325-4c08-91e5-23f3f5d67366 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:13:17.546734 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121dfc86-8325-4c08-91e5-23f3f5d67366 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:13:17.547014 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121dfc86-8325-4c08-91e5-23f3f5d67366 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:13:17.547289 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121dfc86-8325-4c08-91e5-23f3f5d67366 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:13:17.549071 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-121dfc86-8325-4c08-91e5-23f3f5d67366 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:13:17.562723 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121dfc86-8325-4c08-91e5-23f3f5d67366 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:13:17.564176 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121dfc86-8325-4c08-91e5-23f3f5d67366 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:13:17.574936 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1237/2454] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:13:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:13:21.036725 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755e0b8-dc85-4c04-8a81-9b474da2fafe None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:13:21.037271 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755e0b8-dc85-4c04-8a81-9b474da2fafe None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:13:21.037380 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755e0b8-dc85-4c04-8a81-9b474da2fafe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:13:21.055232 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-b755e0b8-dc85-4c04-8a81-9b474da2fafe None None] MFA Rules not processed for user `c710942802df49ec8bd630e4b225eed2`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:13:21.056792 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-b755e0b8-dc85-4c04-8a81-9b474da2fafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:13:21.112322 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1218/2455] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:13:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:13:23.740754 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121dfc86-8325-4c08-91e5-23f3f5d67366 service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:13:23.749771 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121dfc86-8325-4c08-91e5-23f3f5d67366 service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:13:23.753611 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef3b916-8609-43e2-b4bc-96c7a80f3f45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:13:23.754586 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef3b916-8609-43e2-b4bc-96c7a80f3f45 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:13:23.755032 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef3b916-8609-43e2-b4bc-96c7a80f3f45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:13:23.755506 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef3b916-8609-43e2-b4bc-96c7a80f3f45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:13:23.757957 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-aef3b916-8609-43e2-b4bc-96c7a80f3f45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:13:23.770443 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef3b916-8609-43e2-b4bc-96c7a80f3f45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:13:23.771966 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef3b916-8609-43e2-b4bc-96c7a80f3f45 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:13:23.783533 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1238/2456] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:13:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:13:31.105664 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:13:33.778302 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:13:36.812713 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9452f9-c93a-4dc9-b85e-157980e1680d None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:13:36.813238 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9452f9-c93a-4dc9-b85e-157980e1680d None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:13:36.813517 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9452f9-c93a-4dc9-b85e-157980e1680d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:13:36.831600 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-0a9452f9-c93a-4dc9-b85e-157980e1680d None None] MFA Rules not processed for user `86619a80df8841a3ba7d8471ac38056a`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:13:36.833069 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-0a9452f9-c93a-4dc9-b85e-157980e1680d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:13:36.896400 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1219/2457] 10.222.0.13 () {58 vars in 1009 bytes} [Tue Jun 7 22:13:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:13:36.951265 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef3b916-8609-43e2-b4bc-96c7a80f3f45 service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:13:36.952896 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-aef3b916-8609-43e2-b4bc-96c7a80f3f45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:13:37.031807 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef3b916-8609-43e2-b4bc-96c7a80f3f45 service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:13:37.036975 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090dc75b-0d60-4d38-b899-a5318f5f43a9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1261563b6d4e50a4ffeea6a32bdb01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:13:37.037638 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090dc75b-0d60-4d38-b899-a5318f5f43a9 service swift] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:13:37.037921 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090dc75b-0d60-4d38-b899-a5318f5f43a9 service swift] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:13:37.038197 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090dc75b-0d60-4d38-b899-a5318f5f43a9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:13:37.047918 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090dc75b-0d60-4d38-b899-a5318f5f43a9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:13:37.049654 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090dc75b-0d60-4d38-b899-a5318f5f43a9 service swift] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:13:37.058299 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1239/2458] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Jun 7 22:13:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 22:13:39.729381 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9452f9-c93a-4dc9-b85e-157980e1680d None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:13:39.730863 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-0a9452f9-c93a-4dc9-b85e-157980e1680d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:13:39.745972 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9452f9-c93a-4dc9-b85e-157980e1680d None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:13:39.750322 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9295072d-9336-4135-b860-2de152cedd78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:13:39.751036 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295072d-9336-4135-b860-2de152cedd78 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:13:39.751343 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295072d-9336-4135-b860-2de152cedd78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:13:39.751650 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295072d-9336-4135-b860-2de152cedd78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:13:39.755817 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-9295072d-9336-4135-b860-2de152cedd78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:13:39.844752 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9295072d-9336-4135-b860-2de152cedd78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:13:39.846204 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9295072d-9336-4135-b860-2de152cedd78 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:13:39.858021 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1220/2459] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:13:39 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:13:44.868755 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090dc75b-0d60-4d38-b899-a5318f5f43a9 service swift] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:13:44.870450 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-090dc75b-0d60-4d38-b899-a5318f5f43a9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:13:44.891621 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090dc75b-0d60-4d38-b899-a5318f5f43a9 service swift] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:13:44.900473 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2e54fc-3c03-4f6b-b81e-3b255ba1f857 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:13:44.901332 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e54fc-3c03-4f6b-b81e-3b255ba1f857 service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:13:44.901698 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e54fc-3c03-4f6b-b81e-3b255ba1f857 service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:13:44.902046 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e54fc-3c03-4f6b-b81e-3b255ba1f857 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:13:44.905849 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-bd2e54fc-3c03-4f6b-b81e-3b255ba1f857 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:13:45.003823 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2e54fc-3c03-4f6b-b81e-3b255ba1f857 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:13:45.005458 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2e54fc-3c03-4f6b-b81e-3b255ba1f857 service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:13:45.028343 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1240/2460] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:13:44 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:13:49.853024 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=55 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:13:55.024942 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=73 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:13:55.039605 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d2511c-dbe9-4f0a-a8e7-fecfa218ba10 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:13:55.040067 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d2511c-dbe9-4f0a-a8e7-fecfa218ba10 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:13:55.040169 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d2511c-dbe9-4f0a-a8e7-fecfa218ba10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:13:55.057318 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-45d2511c-dbe9-4f0a-a8e7-fecfa218ba10 None None] MFA Rules not processed for user `86619a80df8841a3ba7d8471ac38056a`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:13:55.058502 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-45d2511c-dbe9-4f0a-a8e7-fecfa218ba10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:13:55.066512 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1221/2461] 10.222.0.13 () {58 vars in 1009 bytes} [Tue Jun 7 22:13:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:13:55.078640 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2e54fc-3c03-4f6b-b81e-3b255ba1f857 service nova] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:13:55.089965 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2e54fc-3c03-4f6b-b81e-3b255ba1f857 service nova] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:13:55.095904 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15eb9a2-f50c-4354-a5a6-a51c818b7e3b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1261563b6d4e50a4ffeea6a32bdb01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:13:55.096577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15eb9a2-f50c-4354-a5a6-a51c818b7e3b service swift] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:13:55.096865 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15eb9a2-f50c-4354-a5a6-a51c818b7e3b service swift] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:13:55.097145 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15eb9a2-f50c-4354-a5a6-a51c818b7e3b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:13:55.108332 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15eb9a2-f50c-4354-a5a6-a51c818b7e3b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:13:55.109936 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15eb9a2-f50c-4354-a5a6-a51c818b7e3b service swift] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:13:55.118555 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1241/2462] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Jun 7 22:13:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 22:14:05.052365 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:14:05.114971 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=6 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:14:19.111602 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d2511c-dbe9-4f0a-a8e7-fecfa218ba10 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:19.113988 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-45d2511c-dbe9-4f0a-a8e7-fecfa218ba10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:19.131229 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d2511c-dbe9-4f0a-a8e7-fecfa218ba10 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:19.138590 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283a5143-41b0-479f-b6c8-de741174cd2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:19.139381 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a5143-41b0-479f-b6c8-de741174cd2c service nova] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:19.139732 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a5143-41b0-479f-b6c8-de741174cd2c service nova] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:19.140108 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a5143-41b0-479f-b6c8-de741174cd2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:19.142119 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-283a5143-41b0-479f-b6c8-de741174cd2c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:19.154903 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283a5143-41b0-479f-b6c8-de741174cd2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:19.156706 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283a5143-41b0-479f-b6c8-de741174cd2c service nova] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:19.172078 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1222/2463] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:14:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:14:21.729541 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e2bd8-d2f0-410e-96d4-7af7e337fe74 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:21.729977 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e2bd8-d2f0-410e-96d4-7af7e337fe74 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:21.730087 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e2bd8-d2f0-410e-96d4-7af7e337fe74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:21.747556 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-0b5e2bd8-d2f0-410e-96d4-7af7e337fe74 None None] MFA Rules not processed for user `86619a80df8841a3ba7d8471ac38056a`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:14:21.748751 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-0b5e2bd8-d2f0-410e-96d4-7af7e337fe74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:21.755925 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1242/2464] 10.222.0.13 () {58 vars in 967 bytes} [Tue Jun 7 22:14:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:14:21.767932 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283a5143-41b0-479f-b6c8-de741174cd2c service nova] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:21.779644 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283a5143-41b0-479f-b6c8-de741174cd2c service nova] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:21.785170 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96871fd-7fad-4427-a72e-b0052564dc30 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a1261563b6d4e50a4ffeea6a32bdb01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:21.785801 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96871fd-7fad-4427-a72e-b0052564dc30 service swift] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:21.786091 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96871fd-7fad-4427-a72e-b0052564dc30 service swift] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:21.786370 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96871fd-7fad-4427-a72e-b0052564dc30 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:21.794326 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96871fd-7fad-4427-a72e-b0052564dc30 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:21.795961 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96871fd-7fad-4427-a72e-b0052564dc30 service swift] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:21.804531 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1223/2465] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Jun 7 22:14:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 07 22:14:31.742512 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:14:31.801399 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=12 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:14:37.700860 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e2bd8-d2f0-410e-96d4-7af7e337fe74 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:37.710533 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e2bd8-d2f0-410e-96d4-7af7e337fe74 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:37.714069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af511297-c515-4d6a-afb8-344573962a14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:37.714698 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af511297-c515-4d6a-afb8-344573962a14 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:37.714979 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af511297-c515-4d6a-afb8-344573962a14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:37.715254 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af511297-c515-4d6a-afb8-344573962a14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:37.717039 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-af511297-c515-4d6a-afb8-344573962a14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:37.779191 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af511297-c515-4d6a-afb8-344573962a14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:37.780670 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af511297-c515-4d6a-afb8-344573962a14 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:37.790857 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1243/2466] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:14:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:14:41.075923 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96871fd-7fad-4427-a72e-b0052564dc30 service swift] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:41.077363 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-c96871fd-7fad-4427-a72e-b0052564dc30 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:41.091762 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96871fd-7fad-4427-a72e-b0052564dc30 service swift] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:41.096640 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d5f2f-e947-45af-889d-9deb4dac4c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:41.097654 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d5f2f-e947-45af-889d-9deb4dac4c8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:41.097942 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d5f2f-e947-45af-889d-9deb4dac4c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:41.098223 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d5f2f-e947-45af-889d-9deb4dac4c8d admin admin] PATH_INFO: `/v3/users/0e68eff6a08d4193b3167d47f4e1f97c` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:41.099450 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1d5f2f-e947-45af-889d-9deb4dac4c8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e68eff6a08d4193b3167d47f4e1f97c)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:41.101366 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1d5f2f-e947-45af-889d-9deb4dac4c8d admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:41.150287 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-cd1d5f2f-e947-45af-889d-9deb4dac4c8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e68eff6a08d4193b3167d47f4e1f97c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:41.150940 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-cd1d5f2f-e947-45af-889d-9deb4dac4c8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e68eff6a08d4193b3167d47f4e1f97c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:41.158165 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-cd1d5f2f-e947-45af-889d-9deb4dac4c8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e68eff6a08d4193b3167d47f4e1f97c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:41.170652 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-cd1d5f2f-e947-45af-889d-9deb4dac4c8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e68eff6a08d4193b3167d47f4e1f97c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:41.177711 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-cd1d5f2f-e947-45af-889d-9deb4dac4c8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e68eff6a08d4193b3167d47f4e1f97c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:41.183556 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1224/2467] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:14:41 2022] DELETE /identity/v3/users/0e68eff6a08d4193b3167d47f4e1f97c => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:14:41.375189 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af511297-c515-4d6a-afb8-344573962a14 service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:41.388281 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af511297-c515-4d6a-afb8-344573962a14 service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:41.447576 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4e4339-326f-4582-a0e7-a01f2aa28bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:41.448443 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e4339-326f-4582-a0e7-a01f2aa28bf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:41.448727 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e4339-326f-4582-a0e7-a01f2aa28bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:41.449002 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e4339-326f-4582-a0e7-a01f2aa28bf7 admin admin] PATH_INFO: `/v3/projects/d32f58b7303541388483ca4b881a552e` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:41.450283 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4e4339-326f-4582-a0e7-a01f2aa28bf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d32f58b7303541388483ca4b881a552e)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:41.452236 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4e4339-326f-4582-a0e7-a01f2aa28bf7 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:41.498335 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3b4e4339-326f-4582-a0e7-a01f2aa28bf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd32f58b7303541388483ca4b881a552e', 'request_id': 'req-3b4e4339-326f-4582-a0e7-a01f2aa28bf7'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:41.505646 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3b4e4339-326f-4582-a0e7-a01f2aa28bf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd32f58b7303541388483ca4b881a552e', 'request_id': 'req-3b4e4339-326f-4582-a0e7-a01f2aa28bf7'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:41.511928 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3b4e4339-326f-4582-a0e7-a01f2aa28bf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd32f58b7303541388483ca4b881a552e', 'request_id': 'req-3b4e4339-326f-4582-a0e7-a01f2aa28bf7'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:41.522805 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3b4e4339-326f-4582-a0e7-a01f2aa28bf7 admin admin] The token cache is being invalidate because project d32f58b7303541388483ca4b881a552e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:14:41.523098 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3b4e4339-326f-4582-a0e7-a01f2aa28bf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:41.523863 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3b4e4339-326f-4582-a0e7-a01f2aa28bf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:41.525396 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1244/2468] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:14:41 2022] DELETE /identity/v3/projects/d32f58b7303541388483ca4b881a552e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:14:41.531499 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca5e19-aaa0-45c4-8b12-a1920e0acf17 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:41.531794 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca5e19-aaa0-45c4-8b12-a1920e0acf17 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:41.532076 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca5e19-aaa0-45c4-8b12-a1920e0acf17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:41.550673 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-a1ca5e19-aaa0-45c4-8b12-a1920e0acf17 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:14:41.551931 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-a1ca5e19-aaa0-45c4-8b12-a1920e0acf17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:41.614700 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1225/2469] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:14:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:14:41.622606 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc352cab-1f8a-4986-9808-554c3a8c49f1 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:41.622894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc352cab-1f8a-4986-9808-554c3a8c49f1 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:41.623172 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc352cab-1f8a-4986-9808-554c3a8c49f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:41.641789 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-bc352cab-1f8a-4986-9808-554c3a8c49f1 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:14:41.643509 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-bc352cab-1f8a-4986-9808-554c3a8c49f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:41.651230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1245/2470] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:14:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:14:41.655349 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ca5e19-aaa0-45c4-8b12-a1920e0acf17 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:41.665428 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ca5e19-aaa0-45c4-8b12-a1920e0acf17 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:41.670009 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2866b3-1a37-4896-be48-0b527c458da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:41.670662 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2866b3-1a37-4896-be48-0b527c458da0 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:41.670944 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2866b3-1a37-4896-be48-0b527c458da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:41.671221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2866b3-1a37-4896-be48-0b527c458da0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:41.671989 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2866b3-1a37-4896-be48-0b527c458da0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:14:41.672297 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2866b3-1a37-4896-be48-0b527c458da0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:41.673897 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2866b3-1a37-4896-be48-0b527c458da0 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:41.685924 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1226/2471] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:14:41.690716 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc352cab-1f8a-4986-9808-554c3a8c49f1 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:41.700011 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc352cab-1f8a-4986-9808-554c3a8c49f1 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:41.704715 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d701b8-3d0a-46fe-94a7-09051a4761c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:41.705401 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d701b8-3d0a-46fe-94a7-09051a4761c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:41.705688 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d701b8-3d0a-46fe-94a7-09051a4761c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:41.705965 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d701b8-3d0a-46fe-94a7-09051a4761c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:41.706668 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d701b8-3d0a-46fe-94a7-09051a4761c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:41.708600 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d701b8-3d0a-46fe-94a7-09051a4761c2 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:41.723093 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:14:41.738013 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1246/2472] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:14:41 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:14:41.742168 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2866b3-1a37-4896-be48-0b527c458da0 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:41.752046 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2866b3-1a37-4896-be48-0b527c458da0 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:41.810050 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c5604-e39f-4535-93a0-0992e5e711f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:41.810729 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c5604-e39f-4535-93a0-0992e5e711f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:41.811025 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c5604-e39f-4535-93a0-0992e5e711f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:41.811305 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c5604-e39f-4535-93a0-0992e5e711f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:41.812256 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7c5604-e39f-4535-93a0-0992e5e711f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:41.814126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7c5604-e39f-4535-93a0-0992e5e711f5 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:41.820883 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:14:41.837032 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1227/2473] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:14:41 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:14:41.841079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d701b8-3d0a-46fe-94a7-09051a4761c2 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:41.852101 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d701b8-3d0a-46fe-94a7-09051a4761c2 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:41.857553 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc5c8d-843e-45a9-bfdc-e6468b02ea8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:41.858218 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc5c8d-843e-45a9-bfdc-e6468b02ea8b admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:41.858503 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc5c8d-843e-45a9-bfdc-e6468b02ea8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:41.858781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc5c8d-843e-45a9-bfdc-e6468b02ea8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:41.859569 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fc5c8d-843e-45a9-bfdc-e6468b02ea8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:41.861241 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fc5c8d-843e-45a9-bfdc-e6468b02ea8b admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:41.873439 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1247/2474] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:14:41.876713 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c5604-e39f-4535-93a0-0992e5e711f5 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:41.886836 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c5604-e39f-4535-93a0-0992e5e711f5 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:41.891520 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b916a5a-549d-4ce8-965e-9ca39880b86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:41.892184 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b916a5a-549d-4ce8-965e-9ca39880b86f admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:41.892471 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b916a5a-549d-4ce8-965e-9ca39880b86f admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:41.892772 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b916a5a-549d-4ce8-965e-9ca39880b86f admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:41.893488 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b916a5a-549d-4ce8-965e-9ca39880b86f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:41.895313 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b916a5a-549d-4ce8-965e-9ca39880b86f admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:41.905478 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1228/2475] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:14:41.910661 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc5c8d-843e-45a9-bfdc-e6468b02ea8b admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:41.920523 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc5c8d-843e-45a9-bfdc-e6468b02ea8b admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:41.926644 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aabfad-9810-43dd-af50-cf1116df0909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:41.927563 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aabfad-9810-43dd-af50-cf1116df0909 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:41.927846 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aabfad-9810-43dd-af50-cf1116df0909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:41.928704 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aabfad-9810-43dd-af50-cf1116df0909 admin admin] PATH_INFO: `/v3/projects/541aea07b09a40a2a5309a7eff52d726/users/6940a076b29b48d6ad989dea8b761d5a/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:41.946368 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aabfad-9810-43dd-af50-cf1116df0909 admin admin] RBAC: Authorizing `identity:create_grant(project_id=541aea07b09a40a2a5309a7eff52d726, user_id=6940a076b29b48d6ad989dea8b761d5a, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:41.949152 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aabfad-9810-43dd-af50-cf1116df0909 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:41.964224 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1248/2476] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:14:41 2022] PUT /identity/v3/projects/541aea07b09a40a2a5309a7eff52d726/users/6940a076b29b48d6ad989dea8b761d5a/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:14:41.975567 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad352cb8-f30f-4e51-a3a5-9cac70a53334 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:41.975567 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad352cb8-f30f-4e51-a3a5-9cac70a53334 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:41.975850 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad352cb8-f30f-4e51-a3a5-9cac70a53334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:42.005992 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-ad352cb8-f30f-4e51-a3a5-9cac70a53334 None None] MFA Rules not processed for user `6940a076b29b48d6ad989dea8b761d5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:14:42.007238 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-ad352cb8-f30f-4e51-a3a5-9cac70a53334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:42.081075 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1229/2477] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:14:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:14:42.085773 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aabfad-9810-43dd-af50-cf1116df0909 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:42.150589 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aabfad-9810-43dd-af50-cf1116df0909 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:42.155725 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2808fd-9f81-4122-978e-2806647754c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:42.156517 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2808fd-9f81-4122-978e-2806647754c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:42.156872 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2808fd-9f81-4122-978e-2806647754c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:42.157220 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2808fd-9f81-4122-978e-2806647754c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:42.157982 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2808fd-9f81-4122-978e-2806647754c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:42.160155 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2808fd-9f81-4122-978e-2806647754c3 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:42.186924 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1249/2478] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:14:42 2022] POST /identity/v3/projects => generated 387 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:14:42.192014 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad352cb8-f30f-4e51-a3a5-9cac70a53334 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:42.203066 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad352cb8-f30f-4e51-a3a5-9cac70a53334 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:42.261793 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac96c8a-e1d7-4a9e-8147-3716625000ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:42.262656 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac96c8a-e1d7-4a9e-8147-3716625000ff admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:42.263063 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac96c8a-e1d7-4a9e-8147-3716625000ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:42.263365 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac96c8a-e1d7-4a9e-8147-3716625000ff admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:42.264733 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac96c8a-e1d7-4a9e-8147-3716625000ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:42.266657 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac96c8a-e1d7-4a9e-8147-3716625000ff admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:42.289401 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1230/2479] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:14:42 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:14:42.293694 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2808fd-9f81-4122-978e-2806647754c3 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:42.305672 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2808fd-9f81-4122-978e-2806647754c3 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:42.310394 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab76c3e-e947-45f6-9cc3-ff22c9e62ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:42.311045 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab76c3e-e947-45f6-9cc3-ff22c9e62ce1 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:42.311353 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab76c3e-e947-45f6-9cc3-ff22c9e62ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:42.311636 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab76c3e-e947-45f6-9cc3-ff22c9e62ce1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:42.312373 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab76c3e-e947-45f6-9cc3-ff22c9e62ce1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:42.313980 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab76c3e-e947-45f6-9cc3-ff22c9e62ce1 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:42.326968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1250/2480] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:14:42.332177 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac96c8a-e1d7-4a9e-8147-3716625000ff admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:42.343573 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac96c8a-e1d7-4a9e-8147-3716625000ff admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:42.350489 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2270f66-66b2-47c0-9a55-7e33927b3bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:42.351196 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2270f66-66b2-47c0-9a55-7e33927b3bd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:42.351766 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2270f66-66b2-47c0-9a55-7e33927b3bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:42.352382 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2270f66-66b2-47c0-9a55-7e33927b3bd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:42.353451 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2270f66-66b2-47c0-9a55-7e33927b3bd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:42.355132 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2270f66-66b2-47c0-9a55-7e33927b3bd7 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:42.366722 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1231/2481] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 22:14:42.370946 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab76c3e-e947-45f6-9cc3-ff22c9e62ce1 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:42.381318 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab76c3e-e947-45f6-9cc3-ff22c9e62ce1 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:42.386204 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badee694-95b7-469e-b2a8-b8234baa9479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:42.387091 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badee694-95b7-469e-b2a8-b8234baa9479 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:42.387417 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badee694-95b7-469e-b2a8-b8234baa9479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:42.387703 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badee694-95b7-469e-b2a8-b8234baa9479 admin admin] PATH_INFO: `/v3/projects/29cdb42864ac40a9bae3312fda8045d8/users/269820b759dc476db4f03b6cd642d20a/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:42.404301 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badee694-95b7-469e-b2a8-b8234baa9479 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29cdb42864ac40a9bae3312fda8045d8, user_id=269820b759dc476db4f03b6cd642d20a, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:42.407172 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badee694-95b7-469e-b2a8-b8234baa9479 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:42.421008 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1251/2482] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:14:42 2022] PUT /identity/v3/projects/29cdb42864ac40a9bae3312fda8045d8/users/269820b759dc476db4f03b6cd642d20a/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:14:42.433331 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003d991-8175-4522-a8dd-87e27037ab85 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:42.433716 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003d991-8175-4522-a8dd-87e27037ab85 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:42.434179 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003d991-8175-4522-a8dd-87e27037ab85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:42.460821 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-7003d991-8175-4522-a8dd-87e27037ab85 None None] MFA Rules not processed for user `269820b759dc476db4f03b6cd642d20a`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:14:42.462025 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-7003d991-8175-4522-a8dd-87e27037ab85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:42.496149 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:14:42.528158 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1232/2483] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:14:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:14:42.532570 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badee694-95b7-469e-b2a8-b8234baa9479 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:42.606247 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badee694-95b7-469e-b2a8-b8234baa9479 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:42.615439 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbddad59-e1b6-46a1-9b7f-51b2a8ed4978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:42.616402 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbddad59-e1b6-46a1-9b7f-51b2a8ed4978 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:42.617049 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbddad59-e1b6-46a1-9b7f-51b2a8ed4978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:42.619097 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbddad59-e1b6-46a1-9b7f-51b2a8ed4978 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:42.620115 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbddad59-e1b6-46a1-9b7f-51b2a8ed4978 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:42.622969 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbddad59-e1b6-46a1-9b7f-51b2a8ed4978 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:42.645968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:14:42.661668 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1252/2484] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:14:42 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:14:42.665501 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003d991-8175-4522-a8dd-87e27037ab85 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:42.682373 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003d991-8175-4522-a8dd-87e27037ab85 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:42.756955 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6eb0ec-b0df-4029-ab74-a55c7fa525f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:42.757637 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6eb0ec-b0df-4029-ab74-a55c7fa525f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:42.757921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6eb0ec-b0df-4029-ab74-a55c7fa525f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:42.758196 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6eb0ec-b0df-4029-ab74-a55c7fa525f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:42.760789 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6eb0ec-b0df-4029-ab74-a55c7fa525f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:42.762731 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6eb0ec-b0df-4029-ab74-a55c7fa525f1 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:42.786126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1233/2485] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:14:42 2022] POST /identity/v3/users => generated 299 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:14:42.790113 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbddad59-e1b6-46a1-9b7f-51b2a8ed4978 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:42.801750 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbddad59-e1b6-46a1-9b7f-51b2a8ed4978 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:42.806622 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a21ad52-57cb-4850-adc8-e28c64d77f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:42.807283 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a21ad52-57cb-4850-adc8-e28c64d77f9f admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:42.807599 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a21ad52-57cb-4850-adc8-e28c64d77f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:42.807877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a21ad52-57cb-4850-adc8-e28c64d77f9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:42.808610 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a21ad52-57cb-4850-adc8-e28c64d77f9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:42.810225 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a21ad52-57cb-4850-adc8-e28c64d77f9f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:42.822399 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1253/2486] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:14:42.826716 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6eb0ec-b0df-4029-ab74-a55c7fa525f1 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:42.837866 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6eb0ec-b0df-4029-ab74-a55c7fa525f1 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:42.843209 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e43355-a248-40c1-96ff-47ff771877af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:42.844161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e43355-a248-40c1-96ff-47ff771877af admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:42.844439 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e43355-a248-40c1-96ff-47ff771877af admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:42.844716 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e43355-a248-40c1-96ff-47ff771877af admin admin] PATH_INFO: `/v3/projects/b1d7487cb95049bd86c1df718c093db3/users/120d7c43cc604180b37ba20502e5747a/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:42.863173 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e43355-a248-40c1-96ff-47ff771877af admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1d7487cb95049bd86c1df718c093db3, user_id=120d7c43cc604180b37ba20502e5747a, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:42.865990 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e43355-a248-40c1-96ff-47ff771877af admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:42.881701 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1234/2487] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:14:42 2022] PUT /identity/v3/projects/b1d7487cb95049bd86c1df718c093db3/users/120d7c43cc604180b37ba20502e5747a/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:14:42.885609 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a21ad52-57cb-4850-adc8-e28c64d77f9f admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:42.896157 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a21ad52-57cb-4850-adc8-e28c64d77f9f admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:42.956033 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3adacd6-3ecd-4353-8d7a-0244254a015e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:42.956713 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3adacd6-3ecd-4353-8d7a-0244254a015e admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:42.956985 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3adacd6-3ecd-4353-8d7a-0244254a015e admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:42.957262 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3adacd6-3ecd-4353-8d7a-0244254a015e admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:42.957993 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3adacd6-3ecd-4353-8d7a-0244254a015e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:42.959639 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3adacd6-3ecd-4353-8d7a-0244254a015e admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:42.969912 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1254/2488] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:14:42.974186 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e43355-a248-40c1-96ff-47ff771877af admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:42.985381 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e43355-a248-40c1-96ff-47ff771877af admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:42.990578 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9714d687-4357-484e-84a4-22805ef0f7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:42.991499 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9714d687-4357-484e-84a4-22805ef0f7d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:42.991784 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9714d687-4357-484e-84a4-22805ef0f7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:42.992061 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9714d687-4357-484e-84a4-22805ef0f7d1 admin admin] PATH_INFO: `/v3/projects/b1d7487cb95049bd86c1df718c093db3/users/120d7c43cc604180b37ba20502e5747a/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:42.994019 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9714d687-4357-484e-84a4-22805ef0f7d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1d7487cb95049bd86c1df718c093db3, user_id=120d7c43cc604180b37ba20502e5747a, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:42.996711 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9714d687-4357-484e-84a4-22805ef0f7d1 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:43.007754 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1235/2489] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:14:42 2022] PUT /identity/v3/projects/b1d7487cb95049bd86c1df718c093db3/users/120d7c43cc604180b37ba20502e5747a/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:14:43.018051 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d9a40-251e-4d49-b1b1-bd5d97a295dd None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:43.018343 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d9a40-251e-4d49-b1b1-bd5d97a295dd None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:43.018627 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d9a40-251e-4d49-b1b1-bd5d97a295dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:43.045886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-cd2d9a40-251e-4d49-b1b1-bd5d97a295dd None None] MFA Rules not processed for user `120d7c43cc604180b37ba20502e5747a`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:14:43.047832 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-cd2d9a40-251e-4d49-b1b1-bd5d97a295dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:43.131007 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1255/2490] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:14:43 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:14:43.141867 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9714d687-4357-484e-84a4-22805ef0f7d1 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:43.145105 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-9714d687-4357-484e-84a4-22805ef0f7d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:43.242215 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9714d687-4357-484e-84a4-22805ef0f7d1 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:43.246534 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f6b95-d911-4d79-9eca-857f35e5aa22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:43.247261 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f6b95-d911-4d79-9eca-857f35e5aa22 service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:43.247639 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f6b95-d911-4d79-9eca-857f35e5aa22 service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:43.248011 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f6b95-d911-4d79-9eca-857f35e5aa22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:43.260429 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564f6b95-d911-4d79-9eca-857f35e5aa22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:43.261902 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564f6b95-d911-4d79-9eca-857f35e5aa22 service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:43.274376 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1236/2491] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:14:43 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 22:14:43.804298 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2d9a40-251e-4d49-b1b1-bd5d97a295dd None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:43.805693 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-cd2d9a40-251e-4d49-b1b1-bd5d97a295dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:43.817587 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2d9a40-251e-4d49-b1b1-bd5d97a295dd None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:43.821130 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be34fd62-7513-4424-8b5b-0492886cfd77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:43.821760 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be34fd62-7513-4424-8b5b-0492886cfd77 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:43.822045 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be34fd62-7513-4424-8b5b-0492886cfd77 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:43.822318 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be34fd62-7513-4424-8b5b-0492886cfd77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:43.881966 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be34fd62-7513-4424-8b5b-0492886cfd77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:43.883479 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be34fd62-7513-4424-8b5b-0492886cfd77 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:43.892509 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1256/2492] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:14:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:14:47.870206 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f6b95-d911-4d79-9eca-857f35e5aa22 service cinder] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:47.880550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f6b95-d911-4d79-9eca-857f35e5aa22 service cinder] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:47.884963 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcebe65-3c89-4c55-b999-8c7e38c0650f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:47.885606 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcebe65-3c89-4c55-b999-8c7e38c0650f service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:47.885889 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcebe65-3c89-4c55-b999-8c7e38c0650f service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:47.886168 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcebe65-3c89-4c55-b999-8c7e38c0650f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:47.916735 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:14:47.947949 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcebe65-3c89-4c55-b999-8c7e38c0650f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:47.949382 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcebe65-3c89-4c55-b999-8c7e38c0650f service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:47.958805 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1237/2493] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:14:47 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:14:48.612147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be34fd62-7513-4424-8b5b-0492886cfd77 service cinder] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:48.615014 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-be34fd62-7513-4424-8b5b-0492886cfd77 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:48.621645 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:14:48.671129 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be34fd62-7513-4424-8b5b-0492886cfd77 service cinder] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:48.674738 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd195a6d-46dd-4138-9de4-d3f52f66ec78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:48.676079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd195a6d-46dd-4138-9de4-d3f52f66ec78 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:48.676366 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd195a6d-46dd-4138-9de4-d3f52f66ec78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:48.676641 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd195a6d-46dd-4138-9de4-d3f52f66ec78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:48.678402 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-fd195a6d-46dd-4138-9de4-d3f52f66ec78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:48.741642 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd195a6d-46dd-4138-9de4-d3f52f66ec78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:48.743085 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd195a6d-46dd-4138-9de4-d3f52f66ec78 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:48.752881 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1257/2494] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:14:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:14:50.042346 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcebe65-3c89-4c55-b999-8c7e38c0650f service cinder] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:50.053718 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcebe65-3c89-4c55-b999-8c7e38c0650f service cinder] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:50.058434 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9369aeef-90da-4da6-b536-ea651a47a98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:50.059310 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369aeef-90da-4da6-b536-ea651a47a98f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:50.059624 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369aeef-90da-4da6-b536-ea651a47a98f admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:50.059903 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369aeef-90da-4da6-b536-ea651a47a98f admin admin] PATH_INFO: `/v3/users/6940a076b29b48d6ad989dea8b761d5a` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:50.061126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9369aeef-90da-4da6-b536-ea651a47a98f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6940a076b29b48d6ad989dea8b761d5a)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:50.063049 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9369aeef-90da-4da6-b536-ea651a47a98f admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:50.124872 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-9369aeef-90da-4da6-b536-ea651a47a98f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6940a076b29b48d6ad989dea8b761d5a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:50.125639 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-9369aeef-90da-4da6-b536-ea651a47a98f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6940a076b29b48d6ad989dea8b761d5a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:50.134093 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-9369aeef-90da-4da6-b536-ea651a47a98f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6940a076b29b48d6ad989dea8b761d5a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:50.148459 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-9369aeef-90da-4da6-b536-ea651a47a98f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6940a076b29b48d6ad989dea8b761d5a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:50.156304 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-9369aeef-90da-4da6-b536-ea651a47a98f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6940a076b29b48d6ad989dea8b761d5a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:50.166249 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1238/2495] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:14:50 2022] DELETE /identity/v3/users/6940a076b29b48d6ad989dea8b761d5a => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:14:50.167965 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc30471d-0677-41ae-81e3-0259a0bfe39f None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:50.168451 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc30471d-0677-41ae-81e3-0259a0bfe39f None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:50.168551 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc30471d-0677-41ae-81e3-0259a0bfe39f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:50.187374 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-bc30471d-0677-41ae-81e3-0259a0bfe39f None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:14:50.188612 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-bc30471d-0677-41ae-81e3-0259a0bfe39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:50.275614 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1258/2496] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:14:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:14:50.288698 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9369aeef-90da-4da6-b536-ea651a47a98f admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:50.290490 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-9369aeef-90da-4da6-b536-ea651a47a98f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:50.359421 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9369aeef-90da-4da6-b536-ea651a47a98f admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:50.363442 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16637a47-9453-4dc9-a540-196f05099e89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:50.364191 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16637a47-9453-4dc9-a540-196f05099e89 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:50.364539 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16637a47-9453-4dc9-a540-196f05099e89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:50.364878 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16637a47-9453-4dc9-a540-196f05099e89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:50.375529 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16637a47-9453-4dc9-a540-196f05099e89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:50.377149 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16637a47-9453-4dc9-a540-196f05099e89 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:50.388567 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1239/2497] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:14:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:14:50.703377 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc30471d-0677-41ae-81e3-0259a0bfe39f None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:50.705065 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-bc30471d-0677-41ae-81e3-0259a0bfe39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:50.720133 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc30471d-0677-41ae-81e3-0259a0bfe39f None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:50.729354 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad63be3-717e-46be-86d1-1964dff04546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:50.730263 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad63be3-717e-46be-86d1-1964dff04546 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:50.730546 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad63be3-717e-46be-86d1-1964dff04546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:50.730822 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad63be3-717e-46be-86d1-1964dff04546 admin admin] PATH_INFO: `/v3/projects/541aea07b09a40a2a5309a7eff52d726` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:50.732105 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad63be3-717e-46be-86d1-1964dff04546 admin admin] RBAC: Authorizing `identity:delete_project(project_id=541aea07b09a40a2a5309a7eff52d726)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:50.734067 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad63be3-717e-46be-86d1-1964dff04546 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:50.818354 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3ad63be3-717e-46be-86d1-1964dff04546 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '541aea07b09a40a2a5309a7eff52d726', 'request_id': 'req-3ad63be3-717e-46be-86d1-1964dff04546'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:50.820413 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:14:50.827627 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3ad63be3-717e-46be-86d1-1964dff04546 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '541aea07b09a40a2a5309a7eff52d726', 'request_id': 'req-3ad63be3-717e-46be-86d1-1964dff04546'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:50.835714 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3ad63be3-717e-46be-86d1-1964dff04546 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '541aea07b09a40a2a5309a7eff52d726', 'request_id': 'req-3ad63be3-717e-46be-86d1-1964dff04546'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:50.849249 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3ad63be3-717e-46be-86d1-1964dff04546 admin admin] The token cache is being invalidate because project 541aea07b09a40a2a5309a7eff52d726 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:14:50.849544 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3ad63be3-717e-46be-86d1-1964dff04546 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:50.850348 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3ad63be3-717e-46be-86d1-1964dff04546 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:50.852274 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1259/2498] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:14:50 2022] DELETE /identity/v3/projects/541aea07b09a40a2a5309a7eff52d726 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:14:50.856425 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16637a47-9453-4dc9-a540-196f05099e89 service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:50.868567 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16637a47-9453-4dc9-a540-196f05099e89 service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:50.916035 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:14:50.936020 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdc26de-b960-4731-888c-3042274e3ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:50.937611 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc26de-b960-4731-888c-3042274e3ed8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:50.937975 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc26de-b960-4731-888c-3042274e3ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:50.938261 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc26de-b960-4731-888c-3042274e3ed8 admin admin] PATH_INFO: `/v3/users/269820b759dc476db4f03b6cd642d20a` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:50.939531 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdc26de-b960-4731-888c-3042274e3ed8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=269820b759dc476db4f03b6cd642d20a)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:50.942220 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdc26de-b960-4731-888c-3042274e3ed8 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:50.997865 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-fbdc26de-b960-4731-888c-3042274e3ed8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '269820b759dc476db4f03b6cd642d20a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:50.998516 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-fbdc26de-b960-4731-888c-3042274e3ed8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '269820b759dc476db4f03b6cd642d20a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:51.006701 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-fbdc26de-b960-4731-888c-3042274e3ed8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '269820b759dc476db4f03b6cd642d20a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:51.019191 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-fbdc26de-b960-4731-888c-3042274e3ed8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '269820b759dc476db4f03b6cd642d20a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:51.026666 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-fbdc26de-b960-4731-888c-3042274e3ed8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '269820b759dc476db4f03b6cd642d20a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:51.033194 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1240/2499] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:14:50 2022] DELETE /identity/v3/users/269820b759dc476db4f03b6cd642d20a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:14:51.328832 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad63be3-717e-46be-86d1-1964dff04546 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:51.330941 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-3ad63be3-717e-46be-86d1-1964dff04546 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:51.362547 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad63be3-717e-46be-86d1-1964dff04546 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:51.438864 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856aad1d-fb34-4c91-bead-da11e3fcb978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:51.440475 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856aad1d-fb34-4c91-bead-da11e3fcb978 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:51.442280 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856aad1d-fb34-4c91-bead-da11e3fcb978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:51.442577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856aad1d-fb34-4c91-bead-da11e3fcb978 admin admin] PATH_INFO: `/v3/projects/29cdb42864ac40a9bae3312fda8045d8` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:51.444143 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856aad1d-fb34-4c91-bead-da11e3fcb978 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29cdb42864ac40a9bae3312fda8045d8)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:51.446247 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856aad1d-fb34-4c91-bead-da11e3fcb978 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:51.514993 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-856aad1d-fb34-4c91-bead-da11e3fcb978 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29cdb42864ac40a9bae3312fda8045d8', 'request_id': 'req-856aad1d-fb34-4c91-bead-da11e3fcb978'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:51.532549 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-856aad1d-fb34-4c91-bead-da11e3fcb978 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29cdb42864ac40a9bae3312fda8045d8', 'request_id': 'req-856aad1d-fb34-4c91-bead-da11e3fcb978'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:51.554733 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-856aad1d-fb34-4c91-bead-da11e3fcb978 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29cdb42864ac40a9bae3312fda8045d8', 'request_id': 'req-856aad1d-fb34-4c91-bead-da11e3fcb978'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:51.571802 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-856aad1d-fb34-4c91-bead-da11e3fcb978 admin admin] The token cache is being invalidate because project 29cdb42864ac40a9bae3312fda8045d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:14:51.571802 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-856aad1d-fb34-4c91-bead-da11e3fcb978 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:51.572064 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-856aad1d-fb34-4c91-bead-da11e3fcb978 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:51.574073 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1260/2500] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:14:51 2022] DELETE /identity/v3/projects/29cdb42864ac40a9bae3312fda8045d8 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:14:51.577927 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdc26de-b960-4731-888c-3042274e3ed8 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:51.580211 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-fbdc26de-b960-4731-888c-3042274e3ed8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:51.600126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdc26de-b960-4731-888c-3042274e3ed8 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:51.738864 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6fb88c-4314-4a98-96b5-d541c0b993dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:51.739973 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6fb88c-4314-4a98-96b5-d541c0b993dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:51.740330 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6fb88c-4314-4a98-96b5-d541c0b993dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:51.740840 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6fb88c-4314-4a98-96b5-d541c0b993dc admin admin] PATH_INFO: `/v3/users/120d7c43cc604180b37ba20502e5747a` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:51.742304 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6fb88c-4314-4a98-96b5-d541c0b993dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=120d7c43cc604180b37ba20502e5747a)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:51.744888 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6fb88c-4314-4a98-96b5-d541c0b993dc admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:51.842798 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-da6fb88c-4314-4a98-96b5-d541c0b993dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '120d7c43cc604180b37ba20502e5747a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:51.843803 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-da6fb88c-4314-4a98-96b5-d541c0b993dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '120d7c43cc604180b37ba20502e5747a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:51.856505 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-da6fb88c-4314-4a98-96b5-d541c0b993dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '120d7c43cc604180b37ba20502e5747a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:51.868150 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:14:51.874889 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-da6fb88c-4314-4a98-96b5-d541c0b993dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '120d7c43cc604180b37ba20502e5747a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:51.891273 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-da6fb88c-4314-4a98-96b5-d541c0b993dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '120d7c43cc604180b37ba20502e5747a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:51.904384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1241/2501] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:14:51 2022] DELETE /identity/v3/users/120d7c43cc604180b37ba20502e5747a => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:14:52.170359 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856aad1d-fb34-4c91-bead-da11e3fcb978 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:52.172039 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-856aad1d-fb34-4c91-bead-da11e3fcb978 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:52.187663 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856aad1d-fb34-4c91-bead-da11e3fcb978 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:52.255873 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca51b450-d21b-4354-9746-9ab903a77b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:52.256769 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51b450-d21b-4354-9746-9ab903a77b19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:52.257057 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51b450-d21b-4354-9746-9ab903a77b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:52.257332 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51b450-d21b-4354-9746-9ab903a77b19 admin admin] PATH_INFO: `/v3/users/607b70b4d6e14c95ae2880dd3577c8ae` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:52.262623 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6fb88c-4314-4a98-96b5-d541c0b993dc admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:52.264324 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-da6fb88c-4314-4a98-96b5-d541c0b993dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:52.271395 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:14:52.274213 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca51b450-d21b-4354-9746-9ab903a77b19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=607b70b4d6e14c95ae2880dd3577c8ae)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:52.276489 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca51b450-d21b-4354-9746-9ab903a77b19 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:52.283145 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6fb88c-4314-4a98-96b5-d541c0b993dc admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:52.293405 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462fe40b-f639-48a6-a876-ecb1f1d05d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:52.294295 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462fe40b-f639-48a6-a876-ecb1f1d05d99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:52.294580 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462fe40b-f639-48a6-a876-ecb1f1d05d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:52.294864 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462fe40b-f639-48a6-a876-ecb1f1d05d99 admin admin] PATH_INFO: `/v3/projects/b1d7487cb95049bd86c1df718c093db3` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:52.296788 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462fe40b-f639-48a6-a876-ecb1f1d05d99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1d7487cb95049bd86c1df718c093db3)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:52.299489 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462fe40b-f639-48a6-a876-ecb1f1d05d99 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:52.337643 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-ca51b450-d21b-4354-9746-9ab903a77b19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '607b70b4d6e14c95ae2880dd3577c8ae'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:52.345000 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-ca51b450-d21b-4354-9746-9ab903a77b19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '607b70b4d6e14c95ae2880dd3577c8ae'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:52.353006 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-462fe40b-f639-48a6-a876-ecb1f1d05d99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1d7487cb95049bd86c1df718c093db3', 'request_id': 'req-462fe40b-f639-48a6-a876-ecb1f1d05d99'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:52.359022 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-ca51b450-d21b-4354-9746-9ab903a77b19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '607b70b4d6e14c95ae2880dd3577c8ae'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:52.359999 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-ca51b450-d21b-4354-9746-9ab903a77b19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '607b70b4d6e14c95ae2880dd3577c8ae'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:52.361047 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-ca51b450-d21b-4354-9746-9ab903a77b19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '607b70b4d6e14c95ae2880dd3577c8ae'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:52.361190 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-462fe40b-f639-48a6-a876-ecb1f1d05d99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1d7487cb95049bd86c1df718c093db3', 'request_id': 'req-462fe40b-f639-48a6-a876-ecb1f1d05d99'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:52.368633 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-462fe40b-f639-48a6-a876-ecb1f1d05d99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1d7487cb95049bd86c1df718c093db3', 'request_id': 'req-462fe40b-f639-48a6-a876-ecb1f1d05d99'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:52.376925 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1261/2502] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:14:52 2022] DELETE /identity/v3/users/607b70b4d6e14c95ae2880dd3577c8ae => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:14:52.382437 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-462fe40b-f639-48a6-a876-ecb1f1d05d99 admin admin] The token cache is being invalidate because project b1d7487cb95049bd86c1df718c093db3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:14:52.382805 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-462fe40b-f639-48a6-a876-ecb1f1d05d99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:52.383584 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-462fe40b-f639-48a6-a876-ecb1f1d05d99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:52.385185 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1242/2503] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:14:52 2022] DELETE /identity/v3/projects/b1d7487cb95049bd86c1df718c093db3 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:14:52.391044 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd887490-f9e7-486a-981f-c58f7a208bf2 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:52.391365 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd887490-f9e7-486a-981f-c58f7a208bf2 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:52.391655 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd887490-f9e7-486a-981f-c58f7a208bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:52.410660 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-fd887490-f9e7-486a-981f-c58f7a208bf2 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:14:52.411922 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-fd887490-f9e7-486a-981f-c58f7a208bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:52.475864 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1262/2504] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:14:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:14:52.485336 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980c58a-8f94-493a-8836-b35d5716ad15 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:52.485636 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980c58a-8f94-493a-8836-b35d5716ad15 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:52.485920 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980c58a-8f94-493a-8836-b35d5716ad15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:52.504225 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-1980c58a-8f94-493a-8836-b35d5716ad15 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:14:52.505446 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-1980c58a-8f94-493a-8836-b35d5716ad15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:52.515007 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1243/2505] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:14:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:14:52.520184 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd887490-f9e7-486a-981f-c58f7a208bf2 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:52.530146 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd887490-f9e7-486a-981f-c58f7a208bf2 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:52.535448 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70be3cd-d1a3-4040-a9ab-783068dc8ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:52.536376 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70be3cd-d1a3-4040-a9ab-783068dc8ef0 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:52.536716 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70be3cd-d1a3-4040-a9ab-783068dc8ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:52.537052 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70be3cd-d1a3-4040-a9ab-783068dc8ef0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:52.537841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70be3cd-d1a3-4040-a9ab-783068dc8ef0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:14:52.538211 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70be3cd-d1a3-4040-a9ab-783068dc8ef0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:52.539930 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70be3cd-d1a3-4040-a9ab-783068dc8ef0 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:52.555019 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1980c58a-8f94-493a-8836-b35d5716ad15 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:52.558653 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1263/2506] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:14:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:14:52.558786 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-1980c58a-8f94-493a-8836-b35d5716ad15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:52.559854 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70be3cd-d1a3-4040-a9ab-783068dc8ef0 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:52.569953 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1980c58a-8f94-493a-8836-b35d5716ad15 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:52.574823 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e73a82-63af-4c59-a50a-861a9be3611a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:52.575516 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70be3cd-d1a3-4040-a9ab-783068dc8ef0 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:52.575872 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e73a82-63af-4c59-a50a-861a9be3611a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:52.576224 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e73a82-63af-4c59-a50a-861a9be3611a admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:52.576698 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e73a82-63af-4c59-a50a-861a9be3611a admin admin] PATH_INFO: `/v3/projects/49d73c22e28a450a99d0952aa686dccf` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:52.584921 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671879b-c814-419b-b208-68e001d118e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:52.585592 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671879b-c814-419b-b208-68e001d118e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:52.585881 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671879b-c814-419b-b208-68e001d118e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:52.586187 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671879b-c814-419b-b208-68e001d118e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:52.586922 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f671879b-c814-419b-b208-68e001d118e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:52.588931 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f671879b-c814-419b-b208-68e001d118e7 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:52.589052 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e73a82-63af-4c59-a50a-861a9be3611a admin admin] RBAC: Authorizing `identity:delete_project(project_id=49d73c22e28a450a99d0952aa686dccf)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:52.590866 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e73a82-63af-4c59-a50a-861a9be3611a admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:52.618937 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1264/2507] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:14:52 2022] POST /identity/v3/projects => generated 393 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:14:52.623239 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671879b-c814-419b-b208-68e001d118e7 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:52.642906 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-34e73a82-63af-4c59-a50a-861a9be3611a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49d73c22e28a450a99d0952aa686dccf', 'request_id': 'req-34e73a82-63af-4c59-a50a-861a9be3611a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:52.651546 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-34e73a82-63af-4c59-a50a-861a9be3611a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49d73c22e28a450a99d0952aa686dccf', 'request_id': 'req-34e73a82-63af-4c59-a50a-861a9be3611a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:52.661985 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-34e73a82-63af-4c59-a50a-861a9be3611a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49d73c22e28a450a99d0952aa686dccf', 'request_id': 'req-34e73a82-63af-4c59-a50a-861a9be3611a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:52.671374 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:14:52.672831 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-34e73a82-63af-4c59-a50a-861a9be3611a admin admin] The token cache is being invalidate because project 49d73c22e28a450a99d0952aa686dccf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:14:52.673251 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-34e73a82-63af-4c59-a50a-861a9be3611a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:52.674223 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-34e73a82-63af-4c59-a50a-861a9be3611a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:52.676282 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1244/2508] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:14:52 2022] DELETE /identity/v3/projects/49d73c22e28a450a99d0952aa686dccf => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:14:52.681500 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e488ade-d128-4c00-9b69-fbb47feb9f1b None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:52.681896 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e488ade-d128-4c00-9b69-fbb47feb9f1b None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:52.682276 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e488ade-d128-4c00-9b69-fbb47feb9f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:52.693902 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671879b-c814-419b-b208-68e001d118e7 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:52.700333 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-5e488ade-d128-4c00-9b69-fbb47feb9f1b None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:14:52.701537 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-5e488ade-d128-4c00-9b69-fbb47feb9f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:52.732259 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:14:52.753545 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df7492-c877-4533-b419-fcaf27e7252a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:52.754426 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df7492-c877-4533-b419-fcaf27e7252a admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:52.754600 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df7492-c877-4533-b419-fcaf27e7252a admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:52.754888 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df7492-c877-4533-b419-fcaf27e7252a admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:52.755619 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7df7492-c877-4533-b419-fcaf27e7252a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:52.757566 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7df7492-c877-4533-b419-fcaf27e7252a admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:52.767605 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1245/2509] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:14:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:14:52.775399 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ac760-70de-4110-a69c-31350ded0247 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:52.775693 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ac760-70de-4110-a69c-31350ded0247 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:52.775982 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ac760-70de-4110-a69c-31350ded0247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:52.782066 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1265/2510] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:14:52 2022] POST /identity/v3/users => generated 297 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:14:52.786424 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df7492-c877-4533-b419-fcaf27e7252a admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:52.788015 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a7df7492-c877-4533-b419-fcaf27e7252a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:52.796374 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-e59ac760-70de-4110-a69c-31350ded0247 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:14:52.797566 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-e59ac760-70de-4110-a69c-31350ded0247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:52.799944 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df7492-c877-4533-b419-fcaf27e7252a admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:52.805014 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76904846-f795-420c-a199-98671af9f2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:52.805675 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76904846-f795-420c-a199-98671af9f2e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:52.805795 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1246/2511] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:14:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:14:52.807640 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76904846-f795-420c-a199-98671af9f2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:52.807936 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76904846-f795-420c-a199-98671af9f2e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:52.808937 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76904846-f795-420c-a199-98671af9f2e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:52.809686 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59ac760-70de-4110-a69c-31350ded0247 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:52.810532 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76904846-f795-420c-a199-98671af9f2e0 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:52.819442 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59ac760-70de-4110-a69c-31350ded0247 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:52.824268 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1266/2512] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:14:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 22:14:52.824428 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77133a30-b3dc-443d-8ed9-442b6d01e014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:52.824960 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77133a30-b3dc-443d-8ed9-442b6d01e014 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:52.825193 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76904846-f795-420c-a199-98671af9f2e0 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:52.825301 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77133a30-b3dc-443d-8ed9-442b6d01e014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:52.825513 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77133a30-b3dc-443d-8ed9-442b6d01e014 admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:52.826241 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77133a30-b3dc-443d-8ed9-442b6d01e014 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:14:52.826549 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77133a30-b3dc-443d-8ed9-442b6d01e014 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:52.828175 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77133a30-b3dc-443d-8ed9-442b6d01e014 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:52.835153 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76904846-f795-420c-a199-98671af9f2e0 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:52.839879 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782091a0-5ad5-4dbf-a12b-55f8c057b820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:52.840452 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782091a0-5ad5-4dbf-a12b-55f8c057b820 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:52.840732 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782091a0-5ad5-4dbf-a12b-55f8c057b820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:52.841006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782091a0-5ad5-4dbf-a12b-55f8c057b820 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:52.841720 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782091a0-5ad5-4dbf-a12b-55f8c057b820 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:52.843366 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782091a0-5ad5-4dbf-a12b-55f8c057b820 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:52.843497 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1247/2513] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:14:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 22:14:52.844963 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77133a30-b3dc-443d-8ed9-442b6d01e014 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:52.857173 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1267/2514] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:14:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:14:52.857337 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77133a30-b3dc-443d-8ed9-442b6d01e014 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:52.858167 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782091a0-5ad5-4dbf-a12b-55f8c057b820 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:52.860437 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579edf0e-33ed-4555-b6b4-ac78bfb37284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:52.861082 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579edf0e-33ed-4555-b6b4-ac78bfb37284 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:52.861367 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579edf0e-33ed-4555-b6b4-ac78bfb37284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:52.861645 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579edf0e-33ed-4555-b6b4-ac78bfb37284 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:52.862341 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579edf0e-33ed-4555-b6b4-ac78bfb37284 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:52.864649 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579edf0e-33ed-4555-b6b4-ac78bfb37284 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:52.867598 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782091a0-5ad5-4dbf-a12b-55f8c057b820 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:52.872847 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6212e3-c78d-4580-a05c-a209432fc75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:52.873781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6212e3-c78d-4580-a05c-a209432fc75a admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:52.874066 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6212e3-c78d-4580-a05c-a209432fc75a admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:52.874340 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6212e3-c78d-4580-a05c-a209432fc75a admin admin] PATH_INFO: `/v3/projects/39de195233384321893ad6a06657ffa8/users/80c57f892daa434798bda70c479f6258/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:52.892045 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6212e3-c78d-4580-a05c-a209432fc75a admin admin] RBAC: Authorizing `identity:create_grant(project_id=39de195233384321893ad6a06657ffa8, user_id=80c57f892daa434798bda70c479f6258, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:52.893857 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1248/2515] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:14:52 2022] POST /identity/v3/projects => generated 403 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:14:52.894814 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6212e3-c78d-4580-a05c-a209432fc75a admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:52.897860 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579edf0e-33ed-4555-b6b4-ac78bfb37284 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:52.908640 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1268/2516] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:14:52 2022] PUT /identity/v3/projects/39de195233384321893ad6a06657ffa8/users/80c57f892daa434798bda70c479f6258/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:14:52.920324 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37215e2a-4323-4f8f-b8de-ad487110ffb9 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:52.920611 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37215e2a-4323-4f8f-b8de-ad487110ffb9 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:52.920888 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37215e2a-4323-4f8f-b8de-ad487110ffb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:52.948017 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-37215e2a-4323-4f8f-b8de-ad487110ffb9 None None] MFA Rules not processed for user `80c57f892daa434798bda70c479f6258`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:14:52.949303 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-37215e2a-4323-4f8f-b8de-ad487110ffb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:52.960043 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579edf0e-33ed-4555-b6b4-ac78bfb37284 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:53.015555 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1269/2517] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:14:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:14:53.017558 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df010d-9c29-4435-accc-0e552eaf83a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:53.018229 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df010d-9c29-4435-accc-0e552eaf83a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:53.018515 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df010d-9c29-4435-accc-0e552eaf83a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:53.019013 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df010d-9c29-4435-accc-0e552eaf83a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:53.019669 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3df010d-9c29-4435-accc-0e552eaf83a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:53.019891 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37215e2a-4323-4f8f-b8de-ad487110ffb9 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:53.021579 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3df010d-9c29-4435-accc-0e552eaf83a0 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:53.029966 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37215e2a-4323-4f8f-b8de-ad487110ffb9 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:53.035088 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a8a92-882c-4dc1-8fa2-6e9b0da93d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:53.035798 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a8a92-882c-4dc1-8fa2-6e9b0da93d28 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:53.036078 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a8a92-882c-4dc1-8fa2-6e9b0da93d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:53.036359 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a8a92-882c-4dc1-8fa2-6e9b0da93d28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:53.037030 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701a8a92-882c-4dc1-8fa2-6e9b0da93d28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:53.038911 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701a8a92-882c-4dc1-8fa2-6e9b0da93d28 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:53.042408 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1249/2518] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:14:52 2022] POST /identity/v3/users => generated 302 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:14:53.046459 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df010d-9c29-4435-accc-0e552eaf83a0 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:53.057587 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df010d-9c29-4435-accc-0e552eaf83a0 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:53.060796 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:14:53.063213 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31dfbc1-9e3b-4746-bc72-7838f93a818a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:53.063909 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31dfbc1-9e3b-4746-bc72-7838f93a818a admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:53.064191 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31dfbc1-9e3b-4746-bc72-7838f93a818a admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:53.064468 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31dfbc1-9e3b-4746-bc72-7838f93a818a admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:53.065198 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31dfbc1-9e3b-4746-bc72-7838f93a818a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:53.067078 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31dfbc1-9e3b-4746-bc72-7838f93a818a admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:53.071026 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1270/2519] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:14:53 2022] POST /identity/v3/projects => generated 393 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:14:53.072103 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a8a92-882c-4dc1-8fa2-6e9b0da93d28 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:53.077626 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1250/2520] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:14:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 22:14:53.081236 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31dfbc1-9e3b-4746-bc72-7838f93a818a admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:53.091278 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31dfbc1-9e3b-4746-bc72-7838f93a818a admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:53.104334 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:14:53.136166 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a8a92-882c-4dc1-8fa2-6e9b0da93d28 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:53.141116 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03e5c8c-0cea-476a-9c5d-dc4e2a31f13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:53.141817 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e5c8c-0cea-476a-9c5d-dc4e2a31f13b admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:53.142108 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e5c8c-0cea-476a-9c5d-dc4e2a31f13b admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:53.142382 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e5c8c-0cea-476a-9c5d-dc4e2a31f13b admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:53.143097 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03e5c8c-0cea-476a-9c5d-dc4e2a31f13b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:53.145556 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03e5c8c-0cea-476a-9c5d-dc4e2a31f13b admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:53.151444 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f061979a-9ef5-4485-b7ed-e8f8b44e02b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:53.152107 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061979a-9ef5-4485-b7ed-e8f8b44e02b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:53.152390 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061979a-9ef5-4485-b7ed-e8f8b44e02b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:53.152664 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061979a-9ef5-4485-b7ed-e8f8b44e02b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:53.153416 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f061979a-9ef5-4485-b7ed-e8f8b44e02b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:53.155015 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f061979a-9ef5-4485-b7ed-e8f8b44e02b2 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:53.166544 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1251/2521] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:14:53 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:14:53.166720 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1271/2522] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:14:53 2022] POST /identity/v3/users => generated 303 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:14:53.170377 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f061979a-9ef5-4485-b7ed-e8f8b44e02b2 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:53.173341 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03e5c8c-0cea-476a-9c5d-dc4e2a31f13b admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:53.180864 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f061979a-9ef5-4485-b7ed-e8f8b44e02b2 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:53.184084 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03e5c8c-0cea-476a-9c5d-dc4e2a31f13b admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:53.185582 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056816cb-96f8-465d-a975-674839369665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:53.186443 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056816cb-96f8-465d-a975-674839369665 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:53.186723 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056816cb-96f8-465d-a975-674839369665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:53.186997 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056816cb-96f8-465d-a975-674839369665 admin admin] PATH_INFO: `/v3/projects/f11eb241054042d8a3bd0cc98908adf4/users/6fa954d6b56542dfbfbbc62bc9256495/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:53.189444 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f5ef15-3880-4d10-8865-e08dce04ca26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:53.191054 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f5ef15-3880-4d10-8865-e08dce04ca26 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:53.191592 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f5ef15-3880-4d10-8865-e08dce04ca26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:53.191877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f5ef15-3880-4d10-8865-e08dce04ca26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:53.192611 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f5ef15-3880-4d10-8865-e08dce04ca26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:53.194203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f5ef15-3880-4d10-8865-e08dce04ca26 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:53.201847 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056816cb-96f8-465d-a975-674839369665 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f11eb241054042d8a3bd0cc98908adf4, user_id=6fa954d6b56542dfbfbbc62bc9256495, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:53.204518 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056816cb-96f8-465d-a975-674839369665 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:53.205004 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1272/2523] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:14:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 22:14:53.209659 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f5ef15-3880-4d10-8865-e08dce04ca26 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:53.218834 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1252/2524] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:14:53 2022] PUT /identity/v3/projects/f11eb241054042d8a3bd0cc98908adf4/users/6fa954d6b56542dfbfbbc62bc9256495/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:14:53.219601 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f5ef15-3880-4d10-8865-e08dce04ca26 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:53.224503 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ed598-6817-421b-bfa2-3c2f776c0f87 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:53.224891 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ed598-6817-421b-bfa2-3c2f776c0f87 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:53.225176 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ed598-6817-421b-bfa2-3c2f776c0f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:53.241808 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-5f2ed598-6817-421b-bfa2-3c2f776c0f87 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:14:53.243010 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-5f2ed598-6817-421b-bfa2-3c2f776c0f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:53.288748 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700124dc-c558-4081-8f2f-a73fd406422c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:53.289654 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700124dc-c558-4081-8f2f-a73fd406422c admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:53.289944 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700124dc-c558-4081-8f2f-a73fd406422c admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:53.290223 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700124dc-c558-4081-8f2f-a73fd406422c admin admin] PATH_INFO: `/v3/projects/3b103643ed5a4bec95029c7880f27808/users/dc33da4a14254ae896b36a8100bdf78f/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:53.306199 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700124dc-c558-4081-8f2f-a73fd406422c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b103643ed5a4bec95029c7880f27808, user_id=dc33da4a14254ae896b36a8100bdf78f, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:53.308938 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700124dc-c558-4081-8f2f-a73fd406422c admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:53.319192 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1253/2525] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:14:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:14:53.328779 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1273/2526] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:14:53 2022] PUT /identity/v3/projects/3b103643ed5a4bec95029c7880f27808/users/dc33da4a14254ae896b36a8100bdf78f/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:14:53.330905 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ed598-6817-421b-bfa2-3c2f776c0f87 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:53.333303 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-5f2ed598-6817-421b-bfa2-3c2f776c0f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:53.333447 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700124dc-c558-4081-8f2f-a73fd406422c admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:53.392037 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ed598-6817-421b-bfa2-3c2f776c0f87 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:53.401562 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700124dc-c558-4081-8f2f-a73fd406422c admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:53.405972 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81d8ded-0b8b-4811-aa58-bcb394cf1ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:53.406639 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81d8ded-0b8b-4811-aa58-bcb394cf1ffa admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:53.406926 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81d8ded-0b8b-4811-aa58-bcb394cf1ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:53.407205 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81d8ded-0b8b-4811-aa58-bcb394cf1ffa admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:53.407964 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81d8ded-0b8b-4811-aa58-bcb394cf1ffa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:53.409625 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81d8ded-0b8b-4811-aa58-bcb394cf1ffa admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:53.412342 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:14:53.424752 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1274/2527] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:14:53 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:14:53.426128 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81d8ded-0b8b-4811-aa58-bcb394cf1ffa admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:53.437011 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81d8ded-0b8b-4811-aa58-bcb394cf1ffa admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:53.438112 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:14:53.441702 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5632dec-08e9-40fb-a7f7-13cd562c94d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:53.442555 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5632dec-08e9-40fb-a7f7-13cd562c94d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:53.442832 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5632dec-08e9-40fb-a7f7-13cd562c94d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:53.443106 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5632dec-08e9-40fb-a7f7-13cd562c94d9 admin admin] PATH_INFO: `/v3/projects/3b103643ed5a4bec95029c7880f27808/users/dc33da4a14254ae896b36a8100bdf78f/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:53.444867 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bdba7a-8338-4f44-9e26-64500130908c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:53.445092 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5632dec-08e9-40fb-a7f7-13cd562c94d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b103643ed5a4bec95029c7880f27808, user_id=dc33da4a14254ae896b36a8100bdf78f, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:53.446013 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bdba7a-8338-4f44-9e26-64500130908c service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:53.446528 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bdba7a-8338-4f44-9e26-64500130908c service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:53.447091 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bdba7a-8338-4f44-9e26-64500130908c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:53.447744 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5632dec-08e9-40fb-a7f7-13cd562c94d9 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:53.457131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bdba7a-8338-4f44-9e26-64500130908c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:53.458378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1275/2528] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:14:53 2022] PUT /identity/v3/projects/3b103643ed5a4bec95029c7880f27808/users/dc33da4a14254ae896b36a8100bdf78f/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:14:53.458603 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bdba7a-8338-4f44-9e26-64500130908c service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:53.468303 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f3556-c9ac-44ce-a3eb-c6aee89f4bc7 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:53.468582 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1254/2529] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:14:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:14:53.468706 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f3556-c9ac-44ce-a3eb-c6aee89f4bc7 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:53.468857 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f3556-c9ac-44ce-a3eb-c6aee89f4bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:53.496171 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-928f3556-c9ac-44ce-a3eb-c6aee89f4bc7 None None] MFA Rules not processed for user `dc33da4a14254ae896b36a8100bdf78f`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:14:53.497467 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-928f3556-c9ac-44ce-a3eb-c6aee89f4bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:53.570340 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1276/2530] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:14:53 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:14:53.581139 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bdba7a-8338-4f44-9e26-64500130908c service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:53.583547 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-09bdba7a-8338-4f44-9e26-64500130908c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:53.649726 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bdba7a-8338-4f44-9e26-64500130908c service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:53.701387 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecf88fb-81ca-41e4-95b1-a7a8a1b8b398 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:53.702040 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf88fb-81ca-41e4-95b1-a7a8a1b8b398 service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:53.702328 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf88fb-81ca-41e4-95b1-a7a8a1b8b398 service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:53.702610 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf88fb-81ca-41e4-95b1-a7a8a1b8b398 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:53.714604 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecf88fb-81ca-41e4-95b1-a7a8a1b8b398 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:53.716404 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecf88fb-81ca-41e4-95b1-a7a8a1b8b398 service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:53.727299 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1255/2531] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:14:53 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:14:53.777893 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928f3556-c9ac-44ce-a3eb-c6aee89f4bc7 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:53.779794 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-928f3556-c9ac-44ce-a3eb-c6aee89f4bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:53.795294 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928f3556-c9ac-44ce-a3eb-c6aee89f4bc7 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:53.802814 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb335ea7-b798-48fe-b712-db8431cc4eab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:53.803515 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb335ea7-b798-48fe-b712-db8431cc4eab service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:53.803807 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb335ea7-b798-48fe-b712-db8431cc4eab service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:53.804090 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb335ea7-b798-48fe-b712-db8431cc4eab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:53.878931 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb335ea7-b798-48fe-b712-db8431cc4eab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:53.880619 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb335ea7-b798-48fe-b712-db8431cc4eab service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:53.890329 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1277/2532] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:14:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:14:54.136747 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5b04f-7505-40a2-9408-f420f40290e4 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:54.137184 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5b04f-7505-40a2-9408-f420f40290e4 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:54.137465 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5b04f-7505-40a2-9408-f420f40290e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:54.171627 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-1fd5b04f-7505-40a2-9408-f420f40290e4 None None] MFA Rules not processed for user `6fa954d6b56542dfbfbbc62bc9256495`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:14:54.173398 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-1fd5b04f-7505-40a2-9408-f420f40290e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:54.263348 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1256/2533] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:14:54 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:14:54.268500 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb335ea7-b798-48fe-b712-db8431cc4eab service cinder] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:54.338544 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb335ea7-b798-48fe-b712-db8431cc4eab service cinder] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:54.343125 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c0532-52f4-4f7e-8b14-0d5e5f7dd436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:54.343802 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c0532-52f4-4f7e-8b14-0d5e5f7dd436 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:54.344104 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c0532-52f4-4f7e-8b14-0d5e5f7dd436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:54.344366 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c0532-52f4-4f7e-8b14-0d5e5f7dd436 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:54.345057 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845c0532-52f4-4f7e-8b14-0d5e5f7dd436 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:54.347599 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845c0532-52f4-4f7e-8b14-0d5e5f7dd436 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:54.360802 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:14:54.374392 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1278/2534] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:14:54 2022] POST /identity/v3/projects => generated 405 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:14:54.378474 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5b04f-7505-40a2-9408-f420f40290e4 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:54.389141 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5b04f-7505-40a2-9408-f420f40290e4 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:54.422223 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:14:54.445381 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54c9dcc-ef08-4835-a2f5-96e441557087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:54.446047 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54c9dcc-ef08-4835-a2f5-96e441557087 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:54.446338 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54c9dcc-ef08-4835-a2f5-96e441557087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:54.446604 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54c9dcc-ef08-4835-a2f5-96e441557087 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:54.447281 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54c9dcc-ef08-4835-a2f5-96e441557087 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:54.449234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54c9dcc-ef08-4835-a2f5-96e441557087 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:54.470431 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1257/2535] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:14:54 2022] POST /identity/v3/users => generated 309 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:14:54.475945 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c0532-52f4-4f7e-8b14-0d5e5f7dd436 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:54.486726 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c0532-52f4-4f7e-8b14-0d5e5f7dd436 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:54.492284 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7bbebb-c4c2-4096-846c-7b557b0346bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:54.492937 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7bbebb-c4c2-4096-846c-7b557b0346bd admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:54.493218 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7bbebb-c4c2-4096-846c-7b557b0346bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:54.493491 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7bbebb-c4c2-4096-846c-7b557b0346bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:54.494212 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7bbebb-c4c2-4096-846c-7b557b0346bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:54.495843 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7bbebb-c4c2-4096-846c-7b557b0346bd admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:54.506481 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1279/2536] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:14:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:14:54.510750 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54c9dcc-ef08-4835-a2f5-96e441557087 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:54.520741 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54c9dcc-ef08-4835-a2f5-96e441557087 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:54.525301 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c826e8b9-8aed-4002-ac5b-2930cb18964c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:54.526170 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826e8b9-8aed-4002-ac5b-2930cb18964c admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:54.526452 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826e8b9-8aed-4002-ac5b-2930cb18964c admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:54.526728 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826e8b9-8aed-4002-ac5b-2930cb18964c admin admin] PATH_INFO: `/v3/projects/ca014ef7736242b3adbf262bcbf48943/users/07f35190aa7a4002812053582be72b7c/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:54.541637 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c826e8b9-8aed-4002-ac5b-2930cb18964c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca014ef7736242b3adbf262bcbf48943, user_id=07f35190aa7a4002812053582be72b7c, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:54.544309 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c826e8b9-8aed-4002-ac5b-2930cb18964c admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:54.558144 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1258/2537] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:14:54 2022] PUT /identity/v3/projects/ca014ef7736242b3adbf262bcbf48943/users/07f35190aa7a4002812053582be72b7c/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:14:54.562194 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7bbebb-c4c2-4096-846c-7b557b0346bd admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:54.572200 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7bbebb-c4c2-4096-846c-7b557b0346bd admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:54.631097 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db407495-5fab-4c15-b049-a300a8cdf526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:54.631800 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db407495-5fab-4c15-b049-a300a8cdf526 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:54.632081 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db407495-5fab-4c15-b049-a300a8cdf526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:54.632350 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db407495-5fab-4c15-b049-a300a8cdf526 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:54.633070 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db407495-5fab-4c15-b049-a300a8cdf526 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:54.634647 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db407495-5fab-4c15-b049-a300a8cdf526 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:54.644152 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1280/2538] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:14:54 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:14:54.648497 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c826e8b9-8aed-4002-ac5b-2930cb18964c admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:54.658602 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c826e8b9-8aed-4002-ac5b-2930cb18964c admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:54.663859 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f842a-85a7-4892-b149-7bdd5f88c710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:54.664735 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f842a-85a7-4892-b149-7bdd5f88c710 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:54.665018 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f842a-85a7-4892-b149-7bdd5f88c710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:54.665323 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f842a-85a7-4892-b149-7bdd5f88c710 admin admin] PATH_INFO: `/v3/projects/ca014ef7736242b3adbf262bcbf48943/users/07f35190aa7a4002812053582be72b7c/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:54.667281 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1f842a-85a7-4892-b149-7bdd5f88c710 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca014ef7736242b3adbf262bcbf48943, user_id=07f35190aa7a4002812053582be72b7c, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:54.669917 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1f842a-85a7-4892-b149-7bdd5f88c710 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:54.680589 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1259/2539] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:14:54 2022] PUT /identity/v3/projects/ca014ef7736242b3adbf262bcbf48943/users/07f35190aa7a4002812053582be72b7c/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:14:55.315018 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b3e95-76b2-45d0-a6f2-2b094323cc68 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:55.315608 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b3e95-76b2-45d0-a6f2-2b094323cc68 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:55.316516 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b3e95-76b2-45d0-a6f2-2b094323cc68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:55.345630 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-722b3e95-76b2-45d0-a6f2-2b094323cc68 None None] MFA Rules not processed for user `07f35190aa7a4002812053582be72b7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:14:55.346984 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-722b3e95-76b2-45d0-a6f2-2b094323cc68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:55.427546 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1281/2540] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:14:55 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:14:56.433454 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f842a-85a7-4892-b149-7bdd5f88c710 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:56.435214 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-ea1f842a-85a7-4892-b149-7bdd5f88c710 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:56.492260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722b3e95-76b2-45d0-a6f2-2b094323cc68 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:56.512435 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f842a-85a7-4892-b149-7bdd5f88c710 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:56.517549 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02e03a1-ff92-4d82-a789-859b920521df service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02941c6ad2d54e48b1114d9e030c66c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:56.518281 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e03a1-ff92-4d82-a789-859b920521df service placement] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:56.518767 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e03a1-ff92-4d82-a789-859b920521df service placement] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:56.519458 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e03a1-ff92-4d82-a789-859b920521df service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:56.521561 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-b02e03a1-ff92-4d82-a789-859b920521df service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:56.533234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02e03a1-ff92-4d82-a789-859b920521df service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:56.535801 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02e03a1-ff92-4d82-a789-859b920521df service placement] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:56.545313 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1260/2541] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:14:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:14:56.559611 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722b3e95-76b2-45d0-a6f2-2b094323cc68 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:56.564377 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebccf0f2-3f5a-4fb4-8621-8f538fd42814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:56.565258 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebccf0f2-3f5a-4fb4-8621-8f538fd42814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:56.565543 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebccf0f2-3f5a-4fb4-8621-8f538fd42814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:56.566342 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebccf0f2-3f5a-4fb4-8621-8f538fd42814 admin admin] PATH_INFO: `/v3/users/6fa954d6b56542dfbfbbc62bc9256495` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:56.567586 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebccf0f2-3f5a-4fb4-8621-8f538fd42814 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fa954d6b56542dfbfbbc62bc9256495)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:56.569524 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebccf0f2-3f5a-4fb4-8621-8f538fd42814 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:56.574540 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:14:56.620053 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-ebccf0f2-3f5a-4fb4-8621-8f538fd42814 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fa954d6b56542dfbfbbc62bc9256495'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:56.627045 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-ebccf0f2-3f5a-4fb4-8621-8f538fd42814 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fa954d6b56542dfbfbbc62bc9256495'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:56.639575 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-ebccf0f2-3f5a-4fb4-8621-8f538fd42814 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fa954d6b56542dfbfbbc62bc9256495'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:56.640276 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-ebccf0f2-3f5a-4fb4-8621-8f538fd42814 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fa954d6b56542dfbfbbc62bc9256495'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:56.640869 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-ebccf0f2-3f5a-4fb4-8621-8f538fd42814 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fa954d6b56542dfbfbbc62bc9256495'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:56.654857 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1282/2542] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:14:56 2022] DELETE /identity/v3/users/6fa954d6b56542dfbfbbc62bc9256495 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:14:56.823658 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02e03a1-ff92-4d82-a789-859b920521df service placement] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:56.837273 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02e03a1-ff92-4d82-a789-859b920521df service placement] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:56.893509 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c459a1a-0f42-443f-b2be-294e6ddabbb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:56.894984 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c459a1a-0f42-443f-b2be-294e6ddabbb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:56.895268 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c459a1a-0f42-443f-b2be-294e6ddabbb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:56.895574 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c459a1a-0f42-443f-b2be-294e6ddabbb5 admin admin] PATH_INFO: `/v3/projects/f11eb241054042d8a3bd0cc98908adf4` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:56.896759 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c459a1a-0f42-443f-b2be-294e6ddabbb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f11eb241054042d8a3bd0cc98908adf4)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:56.898657 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c459a1a-0f42-443f-b2be-294e6ddabbb5 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:56.921249 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:14:56.945875 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-2c459a1a-0f42-443f-b2be-294e6ddabbb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f11eb241054042d8a3bd0cc98908adf4', 'request_id': 'req-2c459a1a-0f42-443f-b2be-294e6ddabbb5'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:56.953024 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-2c459a1a-0f42-443f-b2be-294e6ddabbb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f11eb241054042d8a3bd0cc98908adf4', 'request_id': 'req-2c459a1a-0f42-443f-b2be-294e6ddabbb5'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:56.959304 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-2c459a1a-0f42-443f-b2be-294e6ddabbb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f11eb241054042d8a3bd0cc98908adf4', 'request_id': 'req-2c459a1a-0f42-443f-b2be-294e6ddabbb5'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:56.969839 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-2c459a1a-0f42-443f-b2be-294e6ddabbb5 admin admin] The token cache is being invalidate because project f11eb241054042d8a3bd0cc98908adf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:14:56.970127 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-2c459a1a-0f42-443f-b2be-294e6ddabbb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:56.970878 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-2c459a1a-0f42-443f-b2be-294e6ddabbb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:56.972800 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1261/2543] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:14:56 2022] DELETE /identity/v3/projects/f11eb241054042d8a3bd0cc98908adf4 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:14:56.976453 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebccf0f2-3f5a-4fb4-8621-8f538fd42814 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:56.977866 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-ebccf0f2-3f5a-4fb4-8621-8f538fd42814 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:56.992027 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebccf0f2-3f5a-4fb4-8621-8f538fd42814 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:57.045612 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86242af1-363d-4f6a-af9f-d5694e8921d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:57.046475 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86242af1-363d-4f6a-af9f-d5694e8921d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:57.046753 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86242af1-363d-4f6a-af9f-d5694e8921d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:57.047021 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86242af1-363d-4f6a-af9f-d5694e8921d0 admin admin] PATH_INFO: `/v3/users/07f35190aa7a4002812053582be72b7c` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:57.048213 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86242af1-363d-4f6a-af9f-d5694e8921d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07f35190aa7a4002812053582be72b7c)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:57.050628 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86242af1-363d-4f6a-af9f-d5694e8921d0 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:57.094148 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-86242af1-363d-4f6a-af9f-d5694e8921d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07f35190aa7a4002812053582be72b7c'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:57.101547 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-86242af1-363d-4f6a-af9f-d5694e8921d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07f35190aa7a4002812053582be72b7c'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:57.114411 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-86242af1-363d-4f6a-af9f-d5694e8921d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07f35190aa7a4002812053582be72b7c'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:57.115162 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-86242af1-363d-4f6a-af9f-d5694e8921d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07f35190aa7a4002812053582be72b7c'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:57.115882 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-86242af1-363d-4f6a-af9f-d5694e8921d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07f35190aa7a4002812053582be72b7c'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:57.130737 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1283/2544] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:14:56 2022] DELETE /identity/v3/users/07f35190aa7a4002812053582be72b7c => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:14:57.297491 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c459a1a-0f42-443f-b2be-294e6ddabbb5 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:57.299041 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-2c459a1a-0f42-443f-b2be-294e6ddabbb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:57.313958 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c459a1a-0f42-443f-b2be-294e6ddabbb5 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:57.316754 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86242af1-363d-4f6a-af9f-d5694e8921d0 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:57.318261 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-86242af1-363d-4f6a-af9f-d5694e8921d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:57.324630 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:14:57.379536 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763c9089-26fd-4a52-9829-453daa1031b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:57.380413 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c9089-26fd-4a52-9829-453daa1031b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:57.380694 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c9089-26fd-4a52-9829-453daa1031b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:57.380980 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c9089-26fd-4a52-9829-453daa1031b4 admin admin] PATH_INFO: `/v3/users/80c57f892daa434798bda70c479f6258` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:57.382164 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763c9089-26fd-4a52-9829-453daa1031b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80c57f892daa434798bda70c479f6258)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:57.384124 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763c9089-26fd-4a52-9829-453daa1031b4 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:57.392504 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86242af1-363d-4f6a-af9f-d5694e8921d0 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:57.397144 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9afaff6-9297-453b-a903-509e47ad88ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:57.397998 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9afaff6-9297-453b-a903-509e47ad88ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:57.398276 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9afaff6-9297-453b-a903-509e47ad88ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:57.398553 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9afaff6-9297-453b-a903-509e47ad88ac admin admin] PATH_INFO: `/v3/projects/ca014ef7736242b3adbf262bcbf48943` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:57.399780 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9afaff6-9297-453b-a903-509e47ad88ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca014ef7736242b3adbf262bcbf48943)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:57.401673 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9afaff6-9297-453b-a903-509e47ad88ac admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:57.431601 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-763c9089-26fd-4a52-9829-453daa1031b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80c57f892daa434798bda70c479f6258'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:57.432250 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-763c9089-26fd-4a52-9829-453daa1031b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80c57f892daa434798bda70c479f6258'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:57.439490 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-763c9089-26fd-4a52-9829-453daa1031b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80c57f892daa434798bda70c479f6258'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:57.452977 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a9afaff6-9297-453b-a903-509e47ad88ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca014ef7736242b3adbf262bcbf48943', 'request_id': 'req-a9afaff6-9297-453b-a903-509e47ad88ac'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:57.453184 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-763c9089-26fd-4a52-9829-453daa1031b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80c57f892daa434798bda70c479f6258'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:57.460302 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-763c9089-26fd-4a52-9829-453daa1031b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80c57f892daa434798bda70c479f6258'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:57.460441 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a9afaff6-9297-453b-a903-509e47ad88ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca014ef7736242b3adbf262bcbf48943', 'request_id': 'req-a9afaff6-9297-453b-a903-509e47ad88ac'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:57.466337 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a9afaff6-9297-453b-a903-509e47ad88ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca014ef7736242b3adbf262bcbf48943', 'request_id': 'req-a9afaff6-9297-453b-a903-509e47ad88ac'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:57.466590 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1262/2545] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:14:57 2022] DELETE /identity/v3/users/80c57f892daa434798bda70c479f6258 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:14:57.471027 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b3049-4f20-460b-a30c-2a3b6acfce2d None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:57.471216 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b3049-4f20-460b-a30c-2a3b6acfce2d None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:57.471593 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b3049-4f20-460b-a30c-2a3b6acfce2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:57.478876 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a9afaff6-9297-453b-a903-509e47ad88ac admin admin] The token cache is being invalidate because project ca014ef7736242b3adbf262bcbf48943 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:14:57.479166 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a9afaff6-9297-453b-a903-509e47ad88ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:57.479839 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a9afaff6-9297-453b-a903-509e47ad88ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:57.481446 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1284/2546] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:14:57 2022] DELETE /identity/v3/projects/ca014ef7736242b3adbf262bcbf48943 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:14:57.485973 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3b8da-e1fc-468d-a92e-ec59b64a7577 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:57.486262 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3b8da-e1fc-468d-a92e-ec59b64a7577 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:57.486543 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3b8da-e1fc-468d-a92e-ec59b64a7577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:57.489812 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-094b3049-4f20-460b-a30c-2a3b6acfce2d None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:14:57.491009 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-094b3049-4f20-460b-a30c-2a3b6acfce2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:57.505466 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-7ae3b8da-e1fc-468d-a92e-ec59b64a7577 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:14:57.506784 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-7ae3b8da-e1fc-468d-a92e-ec59b64a7577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:57.515828 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:14:57.554919 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1263/2547] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:14:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:14:57.564545 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b3049-4f20-460b-a30c-2a3b6acfce2d None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:57.565981 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-094b3049-4f20-460b-a30c-2a3b6acfce2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:57.568590 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1285/2548] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:14:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:14:57.577061 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1298ab-3f24-444f-86ef-2dacacac9b2b None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:57.577349 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1298ab-3f24-444f-86ef-2dacacac9b2b None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:57.577638 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1298ab-3f24-444f-86ef-2dacacac9b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:57.595177 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-1d1298ab-3f24-444f-86ef-2dacacac9b2b None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:14:57.596402 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-1d1298ab-3f24-444f-86ef-2dacacac9b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:57.605185 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1286/2549] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:14:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:14:57.609852 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1298ab-3f24-444f-86ef-2dacacac9b2b None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:57.619871 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1298ab-3f24-444f-86ef-2dacacac9b2b None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:57.624595 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ada706-cb54-4e40-aaa0-22a2aea61924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:57.624758 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b3049-4f20-460b-a30c-2a3b6acfce2d None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:57.625369 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ada706-cb54-4e40-aaa0-22a2aea61924 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:57.625660 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ada706-cb54-4e40-aaa0-22a2aea61924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:57.625938 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ada706-cb54-4e40-aaa0-22a2aea61924 admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:57.626667 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ada706-cb54-4e40-aaa0-22a2aea61924 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:14:57.626976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ada706-cb54-4e40-aaa0-22a2aea61924 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:57.628418 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3196bc-d60b-4f94-ac02-dcb230b58a7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:57.628821 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ada706-cb54-4e40-aaa0-22a2aea61924 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:57.629820 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3196bc-d60b-4f94-ac02-dcb230b58a7b service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:57.630109 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3196bc-d60b-4f94-ac02-dcb230b58a7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:57.630435 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3196bc-d60b-4f94-ac02-dcb230b58a7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:57.639843 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3196bc-d60b-4f94-ac02-dcb230b58a7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:57.641270 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3196bc-d60b-4f94-ac02-dcb230b58a7b service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:57.641969 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1287/2550] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:14:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:14:57.646117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ada706-cb54-4e40-aaa0-22a2aea61924 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:57.654770 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1264/2551] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:14:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 22:14:57.655662 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ada706-cb54-4e40-aaa0-22a2aea61924 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:57.660810 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0339c76-0cfb-413c-9fef-9f7374f6d688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:57.661467 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0339c76-0cfb-413c-9fef-9f7374f6d688 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:57.661751 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0339c76-0cfb-413c-9fef-9f7374f6d688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:57.662032 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0339c76-0cfb-413c-9fef-9f7374f6d688 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:57.662721 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0339c76-0cfb-413c-9fef-9f7374f6d688 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:57.664666 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0339c76-0cfb-413c-9fef-9f7374f6d688 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:57.675151 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:14:57.699172 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1288/2552] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:14:57 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:14:57.703275 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3196bc-d60b-4f94-ac02-dcb230b58a7b service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:57.713886 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3196bc-d60b-4f94-ac02-dcb230b58a7b service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:57.782171 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5fa009-fced-4f33-a2e4-67ce712a94c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:57.782846 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5fa009-fced-4f33-a2e4-67ce712a94c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:57.783127 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5fa009-fced-4f33-a2e4-67ce712a94c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:57.783431 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5fa009-fced-4f33-a2e4-67ce712a94c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:57.784121 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5fa009-fced-4f33-a2e4-67ce712a94c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:57.787223 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5fa009-fced-4f33-a2e4-67ce712a94c9 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:57.810498 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1265/2553] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:14:57 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:14:57.811949 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0339c76-0cfb-413c-9fef-9f7374f6d688 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:57.823090 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0339c76-0cfb-413c-9fef-9f7374f6d688 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:57.827807 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9a3dc0-c814-4463-bbfc-db8f7b0fcc0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:57.828497 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a3dc0-c814-4463-bbfc-db8f7b0fcc0a admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:57.828781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a3dc0-c814-4463-bbfc-db8f7b0fcc0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:57.829055 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a3dc0-c814-4463-bbfc-db8f7b0fcc0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:57.829781 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9a3dc0-c814-4463-bbfc-db8f7b0fcc0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:57.832050 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9a3dc0-c814-4463-bbfc-db8f7b0fcc0a admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:57.842925 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1289/2554] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:14:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:14:57.847031 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5fa009-fced-4f33-a2e4-67ce712a94c9 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:57.859090 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5fa009-fced-4f33-a2e4-67ce712a94c9 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:57.865004 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c82058d-cad7-43a2-9c34-ed85fee050d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:57.865804 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82058d-cad7-43a2-9c34-ed85fee050d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:57.866174 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82058d-cad7-43a2-9c34-ed85fee050d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:57.866516 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82058d-cad7-43a2-9c34-ed85fee050d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:57.867339 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c82058d-cad7-43a2-9c34-ed85fee050d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:57.868996 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c82058d-cad7-43a2-9c34-ed85fee050d2 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:57.879706 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1266/2555] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:14:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:14:57.883606 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9a3dc0-c814-4463-bbfc-db8f7b0fcc0a admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:57.894761 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9a3dc0-c814-4463-bbfc-db8f7b0fcc0a admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:57.899642 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce1cbb6-7a1a-400a-a52c-0c1073dee2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:57.900557 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1cbb6-7a1a-400a-a52c-0c1073dee2b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:57.900841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1cbb6-7a1a-400a-a52c-0c1073dee2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:57.901116 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1cbb6-7a1a-400a-a52c-0c1073dee2b9 admin admin] PATH_INFO: `/v3/projects/8f990873f5de4a288e6a36005aef8d74/users/f2ba63833d7149f1b90cf370952b1694/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:57.919019 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce1cbb6-7a1a-400a-a52c-0c1073dee2b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f990873f5de4a288e6a36005aef8d74, user_id=f2ba63833d7149f1b90cf370952b1694, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:57.922303 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce1cbb6-7a1a-400a-a52c-0c1073dee2b9 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:57.936080 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c82058d-cad7-43a2-9c34-ed85fee050d2 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:57.937815 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1290/2556] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:14:57 2022] PUT /identity/v3/projects/8f990873f5de4a288e6a36005aef8d74/users/f2ba63833d7149f1b90cf370952b1694/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:14:57.938161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-6c82058d-cad7-43a2-9c34-ed85fee050d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:57.948167 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd023c-8083-47d2-9bc3-8f212bd4b345 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:57.948460 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd023c-8083-47d2-9bc3-8f212bd4b345 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:57.948736 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd023c-8083-47d2-9bc3-8f212bd4b345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:57.969797 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c82058d-cad7-43a2-9c34-ed85fee050d2 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:57.978750 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-bbdd023c-8083-47d2-9bc3-8f212bd4b345 None None] MFA Rules not processed for user `f2ba63833d7149f1b90cf370952b1694`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:14:57.980057 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-bbdd023c-8083-47d2-9bc3-8f212bd4b345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:58.023068 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:14:58.034419 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960a1d5f-ebde-4ec6-b7e8-6452d8668459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:58.035874 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960a1d5f-ebde-4ec6-b7e8-6452d8668459 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:58.036155 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960a1d5f-ebde-4ec6-b7e8-6452d8668459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:58.036911 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960a1d5f-ebde-4ec6-b7e8-6452d8668459 admin admin] PATH_INFO: `/v3/projects/39de195233384321893ad6a06657ffa8` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:58.038119 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960a1d5f-ebde-4ec6-b7e8-6452d8668459 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39de195233384321893ad6a06657ffa8)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:58.040040 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960a1d5f-ebde-4ec6-b7e8-6452d8668459 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:58.045842 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1291/2557] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:14:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:14:58.050166 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdd023c-8083-47d2-9bc3-8f212bd4b345 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:58.060841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdd023c-8083-47d2-9bc3-8f212bd4b345 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:58.065329 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49872fbc-7e48-4eb8-a476-50c5b59004fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:58.066566 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49872fbc-7e48-4eb8-a476-50c5b59004fb admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:58.067388 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49872fbc-7e48-4eb8-a476-50c5b59004fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:58.067674 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49872fbc-7e48-4eb8-a476-50c5b59004fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:58.068357 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49872fbc-7e48-4eb8-a476-50c5b59004fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:58.070430 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49872fbc-7e48-4eb8-a476-50c5b59004fb admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:58.090787 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-960a1d5f-ebde-4ec6-b7e8-6452d8668459 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39de195233384321893ad6a06657ffa8', 'request_id': 'req-960a1d5f-ebde-4ec6-b7e8-6452d8668459'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:58.098305 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1292/2558] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:14:58 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:14:58.099786 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-960a1d5f-ebde-4ec6-b7e8-6452d8668459 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39de195233384321893ad6a06657ffa8', 'request_id': 'req-960a1d5f-ebde-4ec6-b7e8-6452d8668459'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:58.102129 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49872fbc-7e48-4eb8-a476-50c5b59004fb admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:58.106751 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-960a1d5f-ebde-4ec6-b7e8-6452d8668459 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39de195233384321893ad6a06657ffa8', 'request_id': 'req-960a1d5f-ebde-4ec6-b7e8-6452d8668459'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:58.117385 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-960a1d5f-ebde-4ec6-b7e8-6452d8668459 admin admin] The token cache is being invalidate because project 39de195233384321893ad6a06657ffa8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:14:58.117694 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-960a1d5f-ebde-4ec6-b7e8-6452d8668459 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:58.118373 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-960a1d5f-ebde-4ec6-b7e8-6452d8668459 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:58.120189 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1267/2559] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:14:57 2022] DELETE /identity/v3/projects/39de195233384321893ad6a06657ffa8 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:14:58.123730 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960a1d5f-ebde-4ec6-b7e8-6452d8668459 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:58.125328 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-960a1d5f-ebde-4ec6-b7e8-6452d8668459 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:58.165043 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49872fbc-7e48-4eb8-a476-50c5b59004fb admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:58.170117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e684c-7230-4619-97ed-cdebdf4bb285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:58.170787 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e684c-7230-4619-97ed-cdebdf4bb285 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:58.171077 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e684c-7230-4619-97ed-cdebdf4bb285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:58.171383 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e684c-7230-4619-97ed-cdebdf4bb285 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:58.172074 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371e684c-7230-4619-97ed-cdebdf4bb285 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:58.173992 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371e684c-7230-4619-97ed-cdebdf4bb285 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:58.189700 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960a1d5f-ebde-4ec6-b7e8-6452d8668459 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:58.194884 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1293/2560] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:14:58 2022] POST /identity/v3/users => generated 300 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:14:58.195633 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8021725-2422-4051-9f2a-db22609f250c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:58.196502 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8021725-2422-4051-9f2a-db22609f250c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:58.196786 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8021725-2422-4051-9f2a-db22609f250c admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:58.197061 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8021725-2422-4051-9f2a-db22609f250c admin admin] PATH_INFO: `/v3/users/dc33da4a14254ae896b36a8100bdf78f` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:58.198582 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8021725-2422-4051-9f2a-db22609f250c admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc33da4a14254ae896b36a8100bdf78f)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:58.198710 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e684c-7230-4619-97ed-cdebdf4bb285 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:58.200247 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-371e684c-7230-4619-97ed-cdebdf4bb285 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:58.200526 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8021725-2422-4051-9f2a-db22609f250c admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:58.212326 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e684c-7230-4619-97ed-cdebdf4bb285 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:58.217138 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdb6870-bc1a-4407-8154-8666d34433eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:58.217944 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb6870-bc1a-4407-8154-8666d34433eb admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:58.218231 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb6870-bc1a-4407-8154-8666d34433eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:58.218506 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb6870-bc1a-4407-8154-8666d34433eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:58.219267 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdb6870-bc1a-4407-8154-8666d34433eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:58.220932 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdb6870-bc1a-4407-8154-8666d34433eb admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:58.232891 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1294/2561] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:14:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 22:14:58.237035 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdb6870-bc1a-4407-8154-8666d34433eb admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:58.243397 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:14:58.248614 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdb6870-bc1a-4407-8154-8666d34433eb admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:58.255114 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1788695-8d6e-43a5-93a0-72206e38929d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:58.255293 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-e8021725-2422-4051-9f2a-db22609f250c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc33da4a14254ae896b36a8100bdf78f'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:58.256161 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-e8021725-2422-4051-9f2a-db22609f250c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc33da4a14254ae896b36a8100bdf78f'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:58.256316 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1788695-8d6e-43a5-93a0-72206e38929d admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:58.256513 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1788695-8d6e-43a5-93a0-72206e38929d admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:58.257167 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1788695-8d6e-43a5-93a0-72206e38929d admin admin] PATH_INFO: `/v3/projects/bb13abed0b6e47ab91252dfc7ebf00e0/users/cb64eb7dd5934a2e9ec3631badb3fec1/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:58.263578 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-e8021725-2422-4051-9f2a-db22609f250c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc33da4a14254ae896b36a8100bdf78f'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:58.273055 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1788695-8d6e-43a5-93a0-72206e38929d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb13abed0b6e47ab91252dfc7ebf00e0, user_id=cb64eb7dd5934a2e9ec3631badb3fec1, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:58.275767 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1788695-8d6e-43a5-93a0-72206e38929d admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:58.276084 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-e8021725-2422-4051-9f2a-db22609f250c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc33da4a14254ae896b36a8100bdf78f'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:58.283634 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-e8021725-2422-4051-9f2a-db22609f250c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc33da4a14254ae896b36a8100bdf78f'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:58.286722 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:14:58.289424 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1295/2562] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:14:58 2022] PUT /identity/v3/projects/bb13abed0b6e47ab91252dfc7ebf00e0/users/cb64eb7dd5934a2e9ec3631badb3fec1/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:14:58.292140 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1268/2563] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:14:58 2022] DELETE /identity/v3/users/dc33da4a14254ae896b36a8100bdf78f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:14:58.293410 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1788695-8d6e-43a5-93a0-72206e38929d admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:58.295255 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-b1788695-8d6e-43a5-93a0-72206e38929d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:58.360010 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1788695-8d6e-43a5-93a0-72206e38929d admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:58.419754 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e0d7d0-6365-4e9a-a3ec-52da94c9e569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:58.420435 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0d7d0-6365-4e9a-a3ec-52da94c9e569 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:58.420714 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0d7d0-6365-4e9a-a3ec-52da94c9e569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:58.420990 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0d7d0-6365-4e9a-a3ec-52da94c9e569 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:58.421726 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e0d7d0-6365-4e9a-a3ec-52da94c9e569 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:58.423332 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e0d7d0-6365-4e9a-a3ec-52da94c9e569 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:58.436984 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1296/2564] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:14:58 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 22:14:58.438647 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8021725-2422-4051-9f2a-db22609f250c admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:58.455642 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8021725-2422-4051-9f2a-db22609f250c admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:58.461777 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb7fc52-8de2-4685-9779-80aa4f2553bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:58.462755 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb7fc52-8de2-4685-9779-80aa4f2553bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:58.463113 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb7fc52-8de2-4685-9779-80aa4f2553bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:58.463572 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb7fc52-8de2-4685-9779-80aa4f2553bb admin admin] PATH_INFO: `/v3/projects/bb13abed0b6e47ab91252dfc7ebf00e0/users/cb64eb7dd5934a2e9ec3631badb3fec1/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:58.465593 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb7fc52-8de2-4685-9779-80aa4f2553bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb13abed0b6e47ab91252dfc7ebf00e0, user_id=cb64eb7dd5934a2e9ec3631badb3fec1, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:58.468269 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb7fc52-8de2-4685-9779-80aa4f2553bb admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:58.480533 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1269/2565] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:14:58 2022] PUT /identity/v3/projects/bb13abed0b6e47ab91252dfc7ebf00e0/users/cb64eb7dd5934a2e9ec3631badb3fec1/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:14:58.491588 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50471891-6967-42c0-9443-df19f899613d None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:58.491882 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50471891-6967-42c0-9443-df19f899613d None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:58.492159 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50471891-6967-42c0-9443-df19f899613d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:58.518287 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-50471891-6967-42c0-9443-df19f899613d None None] MFA Rules not processed for user `cb64eb7dd5934a2e9ec3631badb3fec1`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:14:58.520566 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-50471891-6967-42c0-9443-df19f899613d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:58.584867 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb7fc52-8de2-4685-9779-80aa4f2553bb admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:58.586805 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-9eb7fc52-8de2-4685-9779-80aa4f2553bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:58.599519 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1297/2566] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:14:58 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:14:58.611041 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50471891-6967-42c0-9443-df19f899613d None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:58.612631 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-50471891-6967-42c0-9443-df19f899613d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:58.623122 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:14:58.664211 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb7fc52-8de2-4685-9779-80aa4f2553bb admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:58.668851 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f99b9-dd2f-4a3e-8c9b-346db09f20d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:58.669724 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f99b9-dd2f-4a3e-8c9b-346db09f20d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:58.670006 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f99b9-dd2f-4a3e-8c9b-346db09f20d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:58.670284 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f99b9-dd2f-4a3e-8c9b-346db09f20d7 admin admin] PATH_INFO: `/v3/projects/3b103643ed5a4bec95029c7880f27808` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:58.671519 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96f99b9-dd2f-4a3e-8c9b-346db09f20d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b103643ed5a4bec95029c7880f27808)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:58.673414 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96f99b9-dd2f-4a3e-8c9b-346db09f20d7 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:58.673530 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50471891-6967-42c0-9443-df19f899613d None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:58.677189 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57784291-2b8b-432f-b4fa-7a3dbcd65955 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:58.677827 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57784291-2b8b-432f-b4fa-7a3dbcd65955 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:58.678106 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57784291-2b8b-432f-b4fa-7a3dbcd65955 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:58.678380 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57784291-2b8b-432f-b4fa-7a3dbcd65955 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:58.687461 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57784291-2b8b-432f-b4fa-7a3dbcd65955 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:58.688888 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57784291-2b8b-432f-b4fa-7a3dbcd65955 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:58.698370 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1298/2567] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:14:58 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:14:58.722064 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c96f99b9-dd2f-4a3e-8c9b-346db09f20d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b103643ed5a4bec95029c7880f27808', 'request_id': 'req-c96f99b9-dd2f-4a3e-8c9b-346db09f20d7'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:58.727439 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57784291-2b8b-432f-b4fa-7a3dbcd65955 service cinder] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:58.733028 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c96f99b9-dd2f-4a3e-8c9b-346db09f20d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b103643ed5a4bec95029c7880f27808', 'request_id': 'req-c96f99b9-dd2f-4a3e-8c9b-346db09f20d7'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:58.742375 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57784291-2b8b-432f-b4fa-7a3dbcd65955 service cinder] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:58.742568 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c96f99b9-dd2f-4a3e-8c9b-346db09f20d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b103643ed5a4bec95029c7880f27808', 'request_id': 'req-c96f99b9-dd2f-4a3e-8c9b-346db09f20d7'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:58.753585 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c96f99b9-dd2f-4a3e-8c9b-346db09f20d7 admin admin] The token cache is being invalidate because project 3b103643ed5a4bec95029c7880f27808 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:14:58.753941 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c96f99b9-dd2f-4a3e-8c9b-346db09f20d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:58.754671 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c96f99b9-dd2f-4a3e-8c9b-346db09f20d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:58.756970 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1270/2568] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:14:58 2022] DELETE /identity/v3/projects/3b103643ed5a4bec95029c7880f27808 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:14:58.762893 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d49db8-8006-41bb-9983-9a998f6f66ed None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:58.763179 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d49db8-8006-41bb-9983-9a998f6f66ed None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:58.763485 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d49db8-8006-41bb-9983-9a998f6f66ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:58.781960 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-c3d49db8-8006-41bb-9983-9a998f6f66ed None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:14:58.783152 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-c3d49db8-8006-41bb-9983-9a998f6f66ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:58.806498 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d734ab-4fbd-47d4-bbf7-75d8aef6b08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:58.807391 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d734ab-4fbd-47d4-bbf7-75d8aef6b08a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:58.807678 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d734ab-4fbd-47d4-bbf7-75d8aef6b08a admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:58.807959 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d734ab-4fbd-47d4-bbf7-75d8aef6b08a admin admin] PATH_INFO: `/v3/users/f2ba63833d7149f1b90cf370952b1694` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:58.809158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d734ab-4fbd-47d4-bbf7-75d8aef6b08a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2ba63833d7149f1b90cf370952b1694)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:58.811067 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d734ab-4fbd-47d4-bbf7-75d8aef6b08a admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:58.846463 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1271/2569] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:14:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:14:58.854528 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36091a-79ba-4fbc-a8c7-0de581360088 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:58.854824 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36091a-79ba-4fbc-a8c7-0de581360088 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:58.855112 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36091a-79ba-4fbc-a8c7-0de581360088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:58.858725 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a6d734ab-4fbd-47d4-bbf7-75d8aef6b08a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2ba63833d7149f1b90cf370952b1694'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:58.866005 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a6d734ab-4fbd-47d4-bbf7-75d8aef6b08a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2ba63833d7149f1b90cf370952b1694'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:58.873124 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-db36091a-79ba-4fbc-a8c7-0de581360088 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:14:58.874330 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-db36091a-79ba-4fbc-a8c7-0de581360088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:58.878035 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a6d734ab-4fbd-47d4-bbf7-75d8aef6b08a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2ba63833d7149f1b90cf370952b1694'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:58.879399 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a6d734ab-4fbd-47d4-bbf7-75d8aef6b08a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2ba63833d7149f1b90cf370952b1694'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:58.880018 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a6d734ab-4fbd-47d4-bbf7-75d8aef6b08a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2ba63833d7149f1b90cf370952b1694'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:58.882850 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1272/2570] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:14:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:14:58.887229 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db36091a-79ba-4fbc-a8c7-0de581360088 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:58.895484 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1299/2571] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:14:58 2022] DELETE /identity/v3/users/f2ba63833d7149f1b90cf370952b1694 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:14:58.897788 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db36091a-79ba-4fbc-a8c7-0de581360088 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:58.899854 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343059b-fd41-40f1-9314-8254ed38443d None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:58.900110 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343059b-fd41-40f1-9314-8254ed38443d None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:58.900391 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343059b-fd41-40f1-9314-8254ed38443d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:58.906119 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:14:58.917939 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-b343059b-fd41-40f1-9314-8254ed38443d None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:14:58.919157 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-b343059b-fd41-40f1-9314-8254ed38443d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:58.944801 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:14:58.966574 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c9976f-bf54-43b7-b8e8-4030c60c733e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:58.967245 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c9976f-bf54-43b7-b8e8-4030c60c733e admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:58.967558 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c9976f-bf54-43b7-b8e8-4030c60c733e admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:58.967834 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c9976f-bf54-43b7-b8e8-4030c60c733e admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:58.968565 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c9976f-bf54-43b7-b8e8-4030c60c733e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:14:58.968870 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c9976f-bf54-43b7-b8e8-4030c60c733e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:58.970470 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c9976f-bf54-43b7-b8e8-4030c60c733e admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:58.986561 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1273/2572] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:14:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 22:14:58.990894 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c9976f-bf54-43b7-b8e8-4030c60c733e admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:58.992593 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-46c9976f-bf54-43b7-b8e8-4030c60c733e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:58.995455 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1300/2573] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:14:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:14:59.005644 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b343059b-fd41-40f1-9314-8254ed38443d None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:59.005802 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c9976f-bf54-43b7-b8e8-4030c60c733e admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:59.009116 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-b343059b-fd41-40f1-9314-8254ed38443d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:59.012126 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a967e6-1d5d-4d30-adc4-1b25f2fd8f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:59.012934 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a967e6-1d5d-4d30-adc4-1b25f2fd8f62 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:59.013219 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a967e6-1d5d-4d30-adc4-1b25f2fd8f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:59.013493 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a967e6-1d5d-4d30-adc4-1b25f2fd8f62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:59.014195 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a967e6-1d5d-4d30-adc4-1b25f2fd8f62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:59.016192 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a967e6-1d5d-4d30-adc4-1b25f2fd8f62 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:59.044008 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1274/2574] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:14:58 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:14:59.048216 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a967e6-1d5d-4d30-adc4-1b25f2fd8f62 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:59.067751 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b343059b-fd41-40f1-9314-8254ed38443d None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:59.109460 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a967e6-1d5d-4d30-adc4-1b25f2fd8f62 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:59.113900 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd1480-6a69-466a-8a4d-f72ff26a529e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:59.114562 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd1480-6a69-466a-8a4d-f72ff26a529e admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:59.114841 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd1480-6a69-466a-8a4d-f72ff26a529e admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:59.115118 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd1480-6a69-466a-8a4d-f72ff26a529e admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:59.115830 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fd1480-6a69-466a-8a4d-f72ff26a529e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:59.116716 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a40c9d-7027-4b09-b281-6fcfd28868e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:59.117367 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a40c9d-7027-4b09-b281-6fcfd28868e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:59.117647 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a40c9d-7027-4b09-b281-6fcfd28868e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:59.117748 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fd1480-6a69-466a-8a4d-f72ff26a529e admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:59.117921 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a40c9d-7027-4b09-b281-6fcfd28868e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:59.127243 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a40c9d-7027-4b09-b281-6fcfd28868e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:59.129611 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a40c9d-7027-4b09-b281-6fcfd28868e3 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:59.138669 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1275/2575] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:14:59 2022] POST /identity/v3/users => generated 283 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:14:59.139726 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1301/2576] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:14:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 22:14:59.142696 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd1480-6a69-466a-8a4d-f72ff26a529e admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:59.152807 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd1480-6a69-466a-8a4d-f72ff26a529e admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:59.157955 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2d08e-0676-4808-ac49-e6aa3f47ed0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:59.158611 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2d08e-0676-4808-ac49-e6aa3f47ed0a admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:59.158891 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2d08e-0676-4808-ac49-e6aa3f47ed0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:59.159163 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2d08e-0676-4808-ac49-e6aa3f47ed0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:59.160293 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e2d08e-0676-4808-ac49-e6aa3f47ed0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:59.163106 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e2d08e-0676-4808-ac49-e6aa3f47ed0a admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:59.173551 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1276/2577] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:14:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 22:14:59.178324 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a40c9d-7027-4b09-b281-6fcfd28868e3 service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:59.188292 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a40c9d-7027-4b09-b281-6fcfd28868e3 service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:59.194388 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f6dfa5-6c03-4b47-bbd8-84831276c321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:59.195175 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6dfa5-6c03-4b47-bbd8-84831276c321 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:59.195499 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6dfa5-6c03-4b47-bbd8-84831276c321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:59.195774 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6dfa5-6c03-4b47-bbd8-84831276c321 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:59.196504 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f6dfa5-6c03-4b47-bbd8-84831276c321 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:59.198254 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f6dfa5-6c03-4b47-bbd8-84831276c321 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:59.209237 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1302/2578] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:14:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:14:59.213664 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2d08e-0676-4808-ac49-e6aa3f47ed0a admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:59.224395 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2d08e-0676-4808-ac49-e6aa3f47ed0a admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:59.229607 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae356dd-00f8-431c-9f69-80a594bff645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:59.230589 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae356dd-00f8-431c-9f69-80a594bff645 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:59.230948 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae356dd-00f8-431c-9f69-80a594bff645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:59.231290 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae356dd-00f8-431c-9f69-80a594bff645 admin admin] PATH_INFO: `/v3/projects/c1e76c3fe4d4497a806ab9f0e36aede0/users/6b0fa59f4a414d03a1a52999566a8892/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:59.247551 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae356dd-00f8-431c-9f69-80a594bff645 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1e76c3fe4d4497a806ab9f0e36aede0, user_id=6b0fa59f4a414d03a1a52999566a8892, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:59.250178 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae356dd-00f8-431c-9f69-80a594bff645 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:59.265619 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1277/2579] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:14:59 2022] PUT /identity/v3/projects/c1e76c3fe4d4497a806ab9f0e36aede0/users/6b0fa59f4a414d03a1a52999566a8892/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:14:59.276133 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5aaab-58db-4ce4-af0a-f57df4c782ee None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:59.276667 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5aaab-58db-4ce4-af0a-f57df4c782ee None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:59.277114 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5aaab-58db-4ce4-af0a-f57df4c782ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:59.308794 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-ebb5aaab-58db-4ce4-af0a-f57df4c782ee None None] MFA Rules not processed for user `6b0fa59f4a414d03a1a52999566a8892`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:14:59.310053 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-ebb5aaab-58db-4ce4-af0a-f57df4c782ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:59.381718 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1303/2580] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:14:59 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:14:59.386403 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae356dd-00f8-431c-9f69-80a594bff645 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:59.417290 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5aaab-58db-4ce4-af0a-f57df4c782ee None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:59.418877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-ebb5aaab-58db-4ce4-af0a-f57df4c782ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:59.429968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:14:59.439233 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:14:59.465033 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae356dd-00f8-431c-9f69-80a594bff645 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:59.471706 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cb90ee-89f9-4d83-a7cd-2d1d84bc90d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:59.472774 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb90ee-89f9-4d83-a7cd-2d1d84bc90d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:59.473221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb90ee-89f9-4d83-a7cd-2d1d84bc90d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:59.473650 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb90ee-89f9-4d83-a7cd-2d1d84bc90d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:59.474562 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cb90ee-89f9-4d83-a7cd-2d1d84bc90d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:59.476735 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cb90ee-89f9-4d83-a7cd-2d1d84bc90d8 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:59.502482 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5aaab-58db-4ce4-af0a-f57df4c782ee None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:59.504286 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1278/2581] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:14:59 2022] POST /identity/v3/projects => generated 365 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:14:59.508180 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cb90ee-89f9-4d83-a7cd-2d1d84bc90d8 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:59.560606 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f7328e-76d1-4312-9fa6-3c85bb13ddb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:59.561526 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7328e-76d1-4312-9fa6-3c85bb13ddb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:59.561756 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7328e-76d1-4312-9fa6-3c85bb13ddb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:59.562035 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7328e-76d1-4312-9fa6-3c85bb13ddb3 admin admin] PATH_INFO: `/v3/projects/8f990873f5de4a288e6a36005aef8d74` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:59.563233 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f7328e-76d1-4312-9fa6-3c85bb13ddb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f990873f5de4a288e6a36005aef8d74)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:59.565236 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f7328e-76d1-4312-9fa6-3c85bb13ddb3 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:59.571664 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cb90ee-89f9-4d83-a7cd-2d1d84bc90d8 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:59.576493 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cb47bd-bae1-4330-9cb1-396227c23b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:59.577164 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb47bd-bae1-4330-9cb1-396227c23b70 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:59.577450 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb47bd-bae1-4330-9cb1-396227c23b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:59.577732 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb47bd-bae1-4330-9cb1-396227c23b70 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:59.578420 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cb47bd-bae1-4330-9cb1-396227c23b70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:59.580379 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cb47bd-bae1-4330-9cb1-396227c23b70 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:59.605707 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1279/2582] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:14:59 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:14:59.607167 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cb47bd-bae1-4330-9cb1-396227c23b70 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:59.615386 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-b7f7328e-76d1-4312-9fa6-3c85bb13ddb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f990873f5de4a288e6a36005aef8d74', 'request_id': 'req-b7f7328e-76d1-4312-9fa6-3c85bb13ddb3'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:59.619964 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cb47bd-bae1-4330-9cb1-396227c23b70 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:59.623024 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-b7f7328e-76d1-4312-9fa6-3c85bb13ddb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f990873f5de4a288e6a36005aef8d74', 'request_id': 'req-b7f7328e-76d1-4312-9fa6-3c85bb13ddb3'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:59.629283 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-b7f7328e-76d1-4312-9fa6-3c85bb13ddb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f990873f5de4a288e6a36005aef8d74', 'request_id': 'req-b7f7328e-76d1-4312-9fa6-3c85bb13ddb3'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:59.640876 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-b7f7328e-76d1-4312-9fa6-3c85bb13ddb3 admin admin] The token cache is being invalidate because project 8f990873f5de4a288e6a36005aef8d74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:14:59.641162 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-b7f7328e-76d1-4312-9fa6-3c85bb13ddb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:59.642539 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-b7f7328e-76d1-4312-9fa6-3c85bb13ddb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:59.644745 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1304/2583] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:14:59 2022] DELETE /identity/v3/projects/8f990873f5de4a288e6a36005aef8d74 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:14:59.648819 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f7328e-76d1-4312-9fa6-3c85bb13ddb3 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:59.650637 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-b7f7328e-76d1-4312-9fa6-3c85bb13ddb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:59.687846 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ce1e7-75eb-4c06-b888-67b15c7f24a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:59.688521 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ce1e7-75eb-4c06-b888-67b15c7f24a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:59.688805 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ce1e7-75eb-4c06-b888-67b15c7f24a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:59.689084 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ce1e7-75eb-4c06-b888-67b15c7f24a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:59.689848 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260ce1e7-75eb-4c06-b888-67b15c7f24a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:59.691749 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260ce1e7-75eb-4c06-b888-67b15c7f24a1 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:59.702308 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1280/2584] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:14:59 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:14:59.706384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ce1e7-75eb-4c06-b888-67b15c7f24a1 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:59.707729 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:14:59.708722 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-260ce1e7-75eb-4c06-b888-67b15c7f24a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:59.724270 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ce1e7-75eb-4c06-b888-67b15c7f24a1 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:59.729348 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcc178e-d8e0-4e59-8863-5f36b22e4bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:59.729948 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f7328e-76d1-4312-9fa6-3c85bb13ddb3 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:59.730267 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc178e-d8e0-4e59-8863-5f36b22e4bad admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:59.730582 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc178e-d8e0-4e59-8863-5f36b22e4bad admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:59.730864 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc178e-d8e0-4e59-8863-5f36b22e4bad admin admin] PATH_INFO: `/v3/projects/34c8d06b29f14c24b82c8393b63c0846/users/a37e71346757447a8b68b6d5e6c0f3ba/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:59.734819 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db190801-96bf-44b3-b30b-66b234a49839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:59.735722 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db190801-96bf-44b3-b30b-66b234a49839 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:59.736007 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db190801-96bf-44b3-b30b-66b234a49839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:59.736287 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db190801-96bf-44b3-b30b-66b234a49839 admin admin] PATH_INFO: `/v3/users/cb64eb7dd5934a2e9ec3631badb3fec1` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:59.737505 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db190801-96bf-44b3-b30b-66b234a49839 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb64eb7dd5934a2e9ec3631badb3fec1)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:59.740507 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db190801-96bf-44b3-b30b-66b234a49839 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:59.746559 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcc178e-d8e0-4e59-8863-5f36b22e4bad admin admin] RBAC: Authorizing `identity:create_grant(project_id=34c8d06b29f14c24b82c8393b63c0846, user_id=a37e71346757447a8b68b6d5e6c0f3ba, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:59.749380 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcc178e-d8e0-4e59-8863-5f36b22e4bad admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:59.767987 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1281/2585] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:14:59 2022] PUT /identity/v3/projects/34c8d06b29f14c24b82c8393b63c0846/users/a37e71346757447a8b68b6d5e6c0f3ba/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:14:59.768196 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcc178e-d8e0-4e59-8863-5f36b22e4bad admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:59.792215 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-db190801-96bf-44b3-b30b-66b234a49839 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb64eb7dd5934a2e9ec3631badb3fec1'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:59.797481 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:14:59.799148 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-db190801-96bf-44b3-b30b-66b234a49839 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb64eb7dd5934a2e9ec3631badb3fec1'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:59.812867 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-db190801-96bf-44b3-b30b-66b234a49839 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb64eb7dd5934a2e9ec3631badb3fec1'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:59.813576 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-db190801-96bf-44b3-b30b-66b234a49839 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb64eb7dd5934a2e9ec3631badb3fec1'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:59.814205 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-db190801-96bf-44b3-b30b-66b234a49839 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb64eb7dd5934a2e9ec3631badb3fec1'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:14:59.828664 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1305/2586] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:14:59 2022] DELETE /identity/v3/users/cb64eb7dd5934a2e9ec3631badb3fec1 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:14:59.836164 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcc178e-d8e0-4e59-8863-5f36b22e4bad admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:59.895291 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6347cc4-f2d6-4756-b9fc-e64a8276e411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:59.895896 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6347cc4-f2d6-4756-b9fc-e64a8276e411 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:59.896182 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6347cc4-f2d6-4756-b9fc-e64a8276e411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:59.896466 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6347cc4-f2d6-4756-b9fc-e64a8276e411 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:59.897196 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6347cc4-f2d6-4756-b9fc-e64a8276e411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:59.898822 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6347cc4-f2d6-4756-b9fc-e64a8276e411 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:59.908870 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1282/2587] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:14:59 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:14:59.913822 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db190801-96bf-44b3-b30b-66b234a49839 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:14:59.915290 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-db190801-96bf-44b3-b30b-66b234a49839 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:14:59.929582 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db190801-96bf-44b3-b30b-66b234a49839 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:14:59.934848 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d30b06-8b70-4c2b-86cc-636bb3ccba81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:14:59.935851 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d30b06-8b70-4c2b-86cc-636bb3ccba81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:59.936201 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d30b06-8b70-4c2b-86cc-636bb3ccba81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:59.936548 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d30b06-8b70-4c2b-86cc-636bb3ccba81 admin admin] PATH_INFO: `/v3/projects/34c8d06b29f14c24b82c8393b63c0846/users/a37e71346757447a8b68b6d5e6c0f3ba/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:59.938638 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d30b06-8b70-4c2b-86cc-636bb3ccba81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34c8d06b29f14c24b82c8393b63c0846, user_id=a37e71346757447a8b68b6d5e6c0f3ba, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:14:59.941420 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d30b06-8b70-4c2b-86cc-636bb3ccba81 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:14:59.952837 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1306/2588] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:14:59 2022] PUT /identity/v3/projects/34c8d06b29f14c24b82c8393b63c0846/users/a37e71346757447a8b68b6d5e6c0f3ba/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:14:59.964794 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104642a5-1ce0-4de9-9d9b-beeb68b3163f None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:14:59.965086 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104642a5-1ce0-4de9-9d9b-beeb68b3163f None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:14:59.965367 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104642a5-1ce0-4de9-9d9b-beeb68b3163f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:14:59.998140 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-104642a5-1ce0-4de9-9d9b-beeb68b3163f None None] MFA Rules not processed for user `a37e71346757447a8b68b6d5e6c0f3ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:14:59.999500 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-104642a5-1ce0-4de9-9d9b-beeb68b3163f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:00.073088 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1283/2589] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:14:59 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:15:00.085953 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d30b06-8b70-4c2b-86cc-636bb3ccba81 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:00.087765 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d6d30b06-8b70-4c2b-86cc-636bb3ccba81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:00.114339 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104642a5-1ce0-4de9-9d9b-beeb68b3163f None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:00.116618 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-104642a5-1ce0-4de9-9d9b-beeb68b3163f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:00.155485 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d30b06-8b70-4c2b-86cc-636bb3ccba81 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:00.159362 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f96a36b-b6f8-4e11-b722-d8d8d8d7e6d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:00.160887 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96a36b-b6f8-4e11-b722-d8d8d8d7e6d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:00.161164 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96a36b-b6f8-4e11-b722-d8d8d8d7e6d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:00.161437 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96a36b-b6f8-4e11-b722-d8d8d8d7e6d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:00.171044 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f96a36b-b6f8-4e11-b722-d8d8d8d7e6d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:00.172537 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f96a36b-b6f8-4e11-b722-d8d8d8d7e6d7 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:00.175951 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:00.182515 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1307/2590] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:15:00 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:15:00.188489 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104642a5-1ce0-4de9-9d9b-beeb68b3163f None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:00.193233 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be26d0-906d-4166-a508-5e448f0d743c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:00.194100 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be26d0-906d-4166-a508-5e448f0d743c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:00.194381 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be26d0-906d-4166-a508-5e448f0d743c admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:00.194658 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be26d0-906d-4166-a508-5e448f0d743c admin admin] PATH_INFO: `/v3/projects/bb13abed0b6e47ab91252dfc7ebf00e0` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:00.195967 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38be26d0-906d-4166-a508-5e448f0d743c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb13abed0b6e47ab91252dfc7ebf00e0)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:00.197844 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38be26d0-906d-4166-a508-5e448f0d743c admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:00.246783 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-38be26d0-906d-4166-a508-5e448f0d743c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb13abed0b6e47ab91252dfc7ebf00e0', 'request_id': 'req-38be26d0-906d-4166-a508-5e448f0d743c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:00.256393 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-38be26d0-906d-4166-a508-5e448f0d743c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb13abed0b6e47ab91252dfc7ebf00e0', 'request_id': 'req-38be26d0-906d-4166-a508-5e448f0d743c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:00.263977 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f96a36b-b6f8-4e11-b722-d8d8d8d7e6d7 service cinder] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:00.268232 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-38be26d0-906d-4166-a508-5e448f0d743c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb13abed0b6e47ab91252dfc7ebf00e0', 'request_id': 'req-38be26d0-906d-4166-a508-5e448f0d743c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:00.274727 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f96a36b-b6f8-4e11-b722-d8d8d8d7e6d7 service cinder] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:00.281223 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-38be26d0-906d-4166-a508-5e448f0d743c admin admin] The token cache is being invalidate because project bb13abed0b6e47ab91252dfc7ebf00e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:15:00.281514 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-38be26d0-906d-4166-a508-5e448f0d743c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:00.282231 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-38be26d0-906d-4166-a508-5e448f0d743c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:00.283859 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1284/2591] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:15:00 2022] DELETE /identity/v3/projects/bb13abed0b6e47ab91252dfc7ebf00e0 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:15:00.289495 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffcd160-2db5-4ae3-95a1-5b8cacf21fdd None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:00.289834 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffcd160-2db5-4ae3-95a1-5b8cacf21fdd None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:00.290116 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffcd160-2db5-4ae3-95a1-5b8cacf21fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:00.308713 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-0ffcd160-2db5-4ae3-95a1-5b8cacf21fdd None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:00.309937 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-0ffcd160-2db5-4ae3-95a1-5b8cacf21fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:00.341667 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef39bda-d2ca-4e0b-827a-33fcd9bec5b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:00.342422 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef39bda-d2ca-4e0b-827a-33fcd9bec5b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:00.342700 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef39bda-d2ca-4e0b-827a-33fcd9bec5b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:00.342976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef39bda-d2ca-4e0b-827a-33fcd9bec5b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:00.345148 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-9ef39bda-d2ca-4e0b-827a-33fcd9bec5b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:00.352877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:00.376216 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1285/2592] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:15:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:15:00.385431 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6ebbf-0ea5-4e89-845c-b0598859875b None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:00.385721 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6ebbf-0ea5-4e89-845c-b0598859875b None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:00.386002 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6ebbf-0ea5-4e89-845c-b0598859875b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:00.405583 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-bad6ebbf-0ea5-4e89-845c-b0598859875b None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:00.407260 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-bad6ebbf-0ea5-4e89-845c-b0598859875b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:00.409217 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef39bda-d2ca-4e0b-827a-33fcd9bec5b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:00.410708 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef39bda-d2ca-4e0b-827a-33fcd9bec5b6 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:00.414691 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1286/2593] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:15:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:15:00.418842 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad6ebbf-0ea5-4e89-845c-b0598859875b None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:00.420132 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1308/2594] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:15:00 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:15:00.428036 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad6ebbf-0ea5-4e89-845c-b0598859875b None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:00.433739 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0d6c6e-d732-4732-add8-bed7b82ca907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:00.435052 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d6c6e-d732-4732-add8-bed7b82ca907 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:00.435357 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d6c6e-d732-4732-add8-bed7b82ca907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:00.435681 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d6c6e-d732-4732-add8-bed7b82ca907 admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:00.436362 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0d6c6e-d732-4732-add8-bed7b82ca907 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:15:00.436686 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0d6c6e-d732-4732-add8-bed7b82ca907 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:00.438269 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0d6c6e-d732-4732-add8-bed7b82ca907 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:00.450710 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1287/2595] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:15:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 22:15:00.454980 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef39bda-d2ca-4e0b-827a-33fcd9bec5b6 service cinder] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:00.466979 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef39bda-d2ca-4e0b-827a-33fcd9bec5b6 service cinder] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:00.472368 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce089a-d4e6-4ae3-9742-58d811c5bec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:00.472963 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce089a-d4e6-4ae3-9742-58d811c5bec2 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:00.473243 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce089a-d4e6-4ae3-9742-58d811c5bec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:00.473516 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce089a-d4e6-4ae3-9742-58d811c5bec2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:00.474201 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ce089a-d4e6-4ae3-9742-58d811c5bec2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:00.476223 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ce089a-d4e6-4ae3-9742-58d811c5bec2 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:00.502921 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1309/2596] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:15:00 2022] POST /identity/v3/projects => generated 401 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:15:00.507879 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0d6c6e-d732-4732-add8-bed7b82ca907 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:00.518148 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0d6c6e-d732-4732-add8-bed7b82ca907 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:00.584079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88646955-9e21-442a-a218-7cea573904df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:00.584792 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88646955-9e21-442a-a218-7cea573904df admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:00.585078 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88646955-9e21-442a-a218-7cea573904df admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:00.585368 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88646955-9e21-442a-a218-7cea573904df admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:00.586051 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88646955-9e21-442a-a218-7cea573904df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:00.588072 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88646955-9e21-442a-a218-7cea573904df admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:00.596773 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:00.613213 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1288/2597] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:15:00 2022] POST /identity/v3/users => generated 301 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:15:00.617769 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce089a-d4e6-4ae3-9742-58d811c5bec2 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:00.629635 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce089a-d4e6-4ae3-9742-58d811c5bec2 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:00.636655 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee87c7d-3f4d-4426-a129-557fe5440644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:00.637475 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee87c7d-3f4d-4426-a129-557fe5440644 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:00.637669 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee87c7d-3f4d-4426-a129-557fe5440644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:00.637909 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee87c7d-3f4d-4426-a129-557fe5440644 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:00.638591 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee87c7d-3f4d-4426-a129-557fe5440644 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:00.640322 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee87c7d-3f4d-4426-a129-557fe5440644 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:00.654195 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1310/2598] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:15:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:15:00.658346 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88646955-9e21-442a-a218-7cea573904df admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:00.673130 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88646955-9e21-442a-a218-7cea573904df admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:00.679121 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d49371c-630a-4de6-b334-63f145fae0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:00.679827 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d49371c-630a-4de6-b334-63f145fae0e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:00.680111 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d49371c-630a-4de6-b334-63f145fae0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:00.680383 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d49371c-630a-4de6-b334-63f145fae0e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:00.681117 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d49371c-630a-4de6-b334-63f145fae0e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:00.682958 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d49371c-630a-4de6-b334-63f145fae0e7 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:00.695362 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1289/2599] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:15:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 22:15:00.698993 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee87c7d-3f4d-4426-a129-557fe5440644 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:00.710693 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee87c7d-3f4d-4426-a129-557fe5440644 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:00.717787 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed97710-4e31-4e5d-83f5-0e99f085e229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:00.719164 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed97710-4e31-4e5d-83f5-0e99f085e229 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:00.719681 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed97710-4e31-4e5d-83f5-0e99f085e229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:00.720146 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed97710-4e31-4e5d-83f5-0e99f085e229 admin admin] PATH_INFO: `/v3/projects/8f5e13ecb54c4181aff9244ada27e46e/users/ed0dc4c366564256b0cbe8b3b20d0e49/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:00.736130 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed97710-4e31-4e5d-83f5-0e99f085e229 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f5e13ecb54c4181aff9244ada27e46e, user_id=ed0dc4c366564256b0cbe8b3b20d0e49, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:00.739368 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed97710-4e31-4e5d-83f5-0e99f085e229 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:00.752711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1311/2600] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:15:00 2022] PUT /identity/v3/projects/8f5e13ecb54c4181aff9244ada27e46e/users/ed0dc4c366564256b0cbe8b3b20d0e49/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:15:00.764054 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b728d-4360-449f-83e2-66c13b774ec6 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:00.764419 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b728d-4360-449f-83e2-66c13b774ec6 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:00.764787 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b728d-4360-449f-83e2-66c13b774ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:00.791260 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-fa5b728d-4360-449f-83e2-66c13b774ec6 None None] MFA Rules not processed for user `ed0dc4c366564256b0cbe8b3b20d0e49`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:00.792524 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-fa5b728d-4360-449f-83e2-66c13b774ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:00.855738 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1290/2601] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:15:00 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:15:00.860239 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed97710-4e31-4e5d-83f5-0e99f085e229 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:00.922573 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed97710-4e31-4e5d-83f5-0e99f085e229 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:00.927837 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925c1c46-0724-4137-b33e-8bbbee4fb11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:00.928491 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925c1c46-0724-4137-b33e-8bbbee4fb11f admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:00.928774 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925c1c46-0724-4137-b33e-8bbbee4fb11f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:00.929048 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925c1c46-0724-4137-b33e-8bbbee4fb11f admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:00.929732 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925c1c46-0724-4137-b33e-8bbbee4fb11f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:00.931661 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925c1c46-0724-4137-b33e-8bbbee4fb11f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:00.956719 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1312/2602] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:15:00 2022] POST /identity/v3/projects => generated 401 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:15:00.960822 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b728d-4360-449f-83e2-66c13b774ec6 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:00.971244 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b728d-4360-449f-83e2-66c13b774ec6 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:01.026417 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9f3f5b-d44f-47d4-8162-45d94322b838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:01.027090 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f3f5b-d44f-47d4-8162-45d94322b838 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:01.027391 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f3f5b-d44f-47d4-8162-45d94322b838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:01.027674 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f3f5b-d44f-47d4-8162-45d94322b838 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:01.028378 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9f3f5b-d44f-47d4-8162-45d94322b838 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:01.030285 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9f3f5b-d44f-47d4-8162-45d94322b838 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:01.051178 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1291/2603] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:15:00 2022] POST /identity/v3/users => generated 307 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:15:01.055212 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925c1c46-0724-4137-b33e-8bbbee4fb11f admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:01.065321 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925c1c46-0724-4137-b33e-8bbbee4fb11f admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:01.070877 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df136f38-42d7-4560-808a-06b064e97536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:01.071576 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df136f38-42d7-4560-808a-06b064e97536 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:01.071886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df136f38-42d7-4560-808a-06b064e97536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:01.072160 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df136f38-42d7-4560-808a-06b064e97536 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:01.072909 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df136f38-42d7-4560-808a-06b064e97536 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:01.074496 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df136f38-42d7-4560-808a-06b064e97536 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:01.085891 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1313/2604] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:15:01.089660 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9f3f5b-d44f-47d4-8162-45d94322b838 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:01.100005 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9f3f5b-d44f-47d4-8162-45d94322b838 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:01.104806 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cdb6a1-4a4f-400e-91fe-93d7cc0374f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:01.105672 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cdb6a1-4a4f-400e-91fe-93d7cc0374f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:01.105952 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cdb6a1-4a4f-400e-91fe-93d7cc0374f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:01.106226 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cdb6a1-4a4f-400e-91fe-93d7cc0374f5 admin admin] PATH_INFO: `/v3/projects/9b201bf93b0a45d8b13dcaf79bcedba1/users/87c0384c2e414572a221399e08af4e2e/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:01.121120 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cdb6a1-4a4f-400e-91fe-93d7cc0374f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b201bf93b0a45d8b13dcaf79bcedba1, user_id=87c0384c2e414572a221399e08af4e2e, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:01.123779 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cdb6a1-4a4f-400e-91fe-93d7cc0374f5 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:01.136297 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1292/2605] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:15:01 2022] PUT /identity/v3/projects/9b201bf93b0a45d8b13dcaf79bcedba1/users/87c0384c2e414572a221399e08af4e2e/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:15:01.140131 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df136f38-42d7-4560-808a-06b064e97536 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:01.150244 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df136f38-42d7-4560-808a-06b064e97536 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:01.170814 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:01.209019 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f430e473-10c7-445f-b5d6-8f66141070f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:01.209614 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f430e473-10c7-445f-b5d6-8f66141070f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:01.209900 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f430e473-10c7-445f-b5d6-8f66141070f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:01.210175 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f430e473-10c7-445f-b5d6-8f66141070f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:01.210898 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f430e473-10c7-445f-b5d6-8f66141070f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:01.212517 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f430e473-10c7-445f-b5d6-8f66141070f0 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:01.222627 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1314/2606] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:15:01.226772 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cdb6a1-4a4f-400e-91fe-93d7cc0374f5 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:01.237216 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cdb6a1-4a4f-400e-91fe-93d7cc0374f5 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:01.242616 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b94a54-6470-4ef8-9fd9-bb5816c092d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:01.243528 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b94a54-6470-4ef8-9fd9-bb5816c092d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:01.243813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b94a54-6470-4ef8-9fd9-bb5816c092d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:01.244091 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b94a54-6470-4ef8-9fd9-bb5816c092d3 admin admin] PATH_INFO: `/v3/projects/9b201bf93b0a45d8b13dcaf79bcedba1/users/87c0384c2e414572a221399e08af4e2e/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:01.246023 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b94a54-6470-4ef8-9fd9-bb5816c092d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b201bf93b0a45d8b13dcaf79bcedba1, user_id=87c0384c2e414572a221399e08af4e2e, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:01.248724 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b94a54-6470-4ef8-9fd9-bb5816c092d3 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:01.258962 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1293/2607] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:15:01 2022] PUT /identity/v3/projects/9b201bf93b0a45d8b13dcaf79bcedba1/users/87c0384c2e414572a221399e08af4e2e/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:15:01.270168 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f04b8-58c7-4ed0-a9e2-ce707ee13b04 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:01.270460 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f04b8-58c7-4ed0-a9e2-ce707ee13b04 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:01.270735 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f04b8-58c7-4ed0-a9e2-ce707ee13b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:01.297388 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-8f6f04b8-58c7-4ed0-a9e2-ce707ee13b04 None None] MFA Rules not processed for user `87c0384c2e414572a221399e08af4e2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:01.298582 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-8f6f04b8-58c7-4ed0-a9e2-ce707ee13b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:01.365832 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1315/2608] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:15:01 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:15:01.376935 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b94a54-6470-4ef8-9fd9-bb5816c092d3 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:01.378857 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-78b94a54-6470-4ef8-9fd9-bb5816c092d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:01.406046 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:01.436279 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b94a54-6470-4ef8-9fd9-bb5816c092d3 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:01.439858 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837123e3-3bf6-4613-8ac6-c436cb736437 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:01.440524 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837123e3-3bf6-4613-8ac6-c436cb736437 service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:01.440948 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837123e3-3bf6-4613-8ac6-c436cb736437 service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:01.441273 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837123e3-3bf6-4613-8ac6-c436cb736437 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:01.450023 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837123e3-3bf6-4613-8ac6-c436cb736437 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:01.451473 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837123e3-3bf6-4613-8ac6-c436cb736437 service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:01.461966 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1294/2609] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:15:01 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:15:01.532113 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6f04b8-58c7-4ed0-a9e2-ce707ee13b04 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:01.542469 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6f04b8-58c7-4ed0-a9e2-ce707ee13b04 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:01.546114 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b44cb-10b3-4a67-af25-8fb52f1a1117 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:01.546749 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b44cb-10b3-4a67-af25-8fb52f1a1117 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:01.547033 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b44cb-10b3-4a67-af25-8fb52f1a1117 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:01.547313 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b44cb-10b3-4a67-af25-8fb52f1a1117 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:01.603090 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48b44cb-10b3-4a67-af25-8fb52f1a1117 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:01.604810 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48b44cb-10b3-4a67-af25-8fb52f1a1117 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:01.614645 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1316/2610] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:15:01 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 22:15:05.137905 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837123e3-3bf6-4613-8ac6-c436cb736437 service cinder] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:05.139311 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-837123e3-3bf6-4613-8ac6-c436cb736437 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:05.205718 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837123e3-3bf6-4613-8ac6-c436cb736437 service cinder] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:05.210859 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50498961-68f6-4bbd-8cb8-83a8476d6f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:05.211754 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50498961-68f6-4bbd-8cb8-83a8476d6f97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:05.212037 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50498961-68f6-4bbd-8cb8-83a8476d6f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:05.212311 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50498961-68f6-4bbd-8cb8-83a8476d6f97 admin admin] PATH_INFO: `/v3/users/6b0fa59f4a414d03a1a52999566a8892` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:05.213481 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50498961-68f6-4bbd-8cb8-83a8476d6f97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b0fa59f4a414d03a1a52999566a8892)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:05.215397 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50498961-68f6-4bbd-8cb8-83a8476d6f97 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:05.265892 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-50498961-68f6-4bbd-8cb8-83a8476d6f97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b0fa59f4a414d03a1a52999566a8892'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:05.266539 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-50498961-68f6-4bbd-8cb8-83a8476d6f97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b0fa59f4a414d03a1a52999566a8892'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:05.274184 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-50498961-68f6-4bbd-8cb8-83a8476d6f97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b0fa59f4a414d03a1a52999566a8892'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:05.286366 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-50498961-68f6-4bbd-8cb8-83a8476d6f97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b0fa59f4a414d03a1a52999566a8892'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:05.293146 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-50498961-68f6-4bbd-8cb8-83a8476d6f97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b0fa59f4a414d03a1a52999566a8892'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:05.299194 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1295/2611] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:15:05 2022] DELETE /identity/v3/users/6b0fa59f4a414d03a1a52999566a8892 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:15:05.307009 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f7df4e-b92e-4f53-942a-31d31d17d865 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:05.307531 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f7df4e-b92e-4f53-942a-31d31d17d865 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:05.307531 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f7df4e-b92e-4f53-942a-31d31d17d865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:05.325241 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-02f7df4e-b92e-4f53-942a-31d31d17d865 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:05.326423 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-02f7df4e-b92e-4f53-942a-31d31d17d865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:05.365615 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:05.385201 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1317/2612] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:15:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:15:05.394694 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50498961-68f6-4bbd-8cb8-83a8476d6f97 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:05.396222 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-50498961-68f6-4bbd-8cb8-83a8476d6f97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:05.448515 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:05.454684 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50498961-68f6-4bbd-8cb8-83a8476d6f97 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:05.459505 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee88bd-4832-45d5-98fc-d22f2017a365 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:05.460153 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee88bd-4832-45d5-98fc-d22f2017a365 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:05.460437 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee88bd-4832-45d5-98fc-d22f2017a365 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:05.460739 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee88bd-4832-45d5-98fc-d22f2017a365 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:05.469690 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dee88bd-4832-45d5-98fc-d22f2017a365 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:05.471104 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dee88bd-4832-45d5-98fc-d22f2017a365 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:05.480694 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1296/2613] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:15:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:15:05.677746 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f7df4e-b92e-4f53-942a-31d31d17d865 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:05.679188 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-02f7df4e-b92e-4f53-942a-31d31d17d865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:05.690985 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f7df4e-b92e-4f53-942a-31d31d17d865 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:05.695870 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d9c97a-505f-4ccb-a177-f433885e3f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:05.696825 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d9c97a-505f-4ccb-a177-f433885e3f67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:05.697196 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d9c97a-505f-4ccb-a177-f433885e3f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:05.697534 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d9c97a-505f-4ccb-a177-f433885e3f67 admin admin] PATH_INFO: `/v3/users/ed0dc4c366564256b0cbe8b3b20d0e49` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:05.698799 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d9c97a-505f-4ccb-a177-f433885e3f67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed0dc4c366564256b0cbe8b3b20d0e49)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:05.700901 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d9c97a-505f-4ccb-a177-f433885e3f67 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:05.762481 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-01d9c97a-505f-4ccb-a177-f433885e3f67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed0dc4c366564256b0cbe8b3b20d0e49'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:05.768818 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee88bd-4832-45d5-98fc-d22f2017a365 service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:05.771264 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-3dee88bd-4832-45d5-98fc-d22f2017a365 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:05.774666 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-01d9c97a-505f-4ccb-a177-f433885e3f67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed0dc4c366564256b0cbe8b3b20d0e49'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:05.784034 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee88bd-4832-45d5-98fc-d22f2017a365 service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:05.789525 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810c2a60-ec25-46b8-aef3-8fa1f522b1f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:05.790204 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-01d9c97a-505f-4ccb-a177-f433885e3f67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed0dc4c366564256b0cbe8b3b20d0e49'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:05.790505 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c2a60-ec25-46b8-aef3-8fa1f522b1f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:05.790866 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c2a60-ec25-46b8-aef3-8fa1f522b1f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:05.791250 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c2a60-ec25-46b8-aef3-8fa1f522b1f2 admin admin] PATH_INFO: `/v3/projects/c1e76c3fe4d4497a806ab9f0e36aede0` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:05.791417 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-01d9c97a-505f-4ccb-a177-f433885e3f67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed0dc4c366564256b0cbe8b3b20d0e49'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:05.792088 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-01d9c97a-505f-4ccb-a177-f433885e3f67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed0dc4c366564256b0cbe8b3b20d0e49'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:05.792611 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810c2a60-ec25-46b8-aef3-8fa1f522b1f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1e76c3fe4d4497a806ab9f0e36aede0)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:05.794618 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810c2a60-ec25-46b8-aef3-8fa1f522b1f2 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:05.808092 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1318/2614] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:15:05 2022] DELETE /identity/v3/users/ed0dc4c366564256b0cbe8b3b20d0e49 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:15:05.813378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec5545-c4e0-4c26-905c-7ed2b3cff38a None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:05.813669 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec5545-c4e0-4c26-905c-7ed2b3cff38a None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:05.813977 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec5545-c4e0-4c26-905c-7ed2b3cff38a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:05.832349 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-08ec5545-c4e0-4c26-905c-7ed2b3cff38a None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:05.833585 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-08ec5545-c4e0-4c26-905c-7ed2b3cff38a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:05.844667 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-810c2a60-ec25-46b8-aef3-8fa1f522b1f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1e76c3fe4d4497a806ab9f0e36aede0', 'request_id': 'req-810c2a60-ec25-46b8-aef3-8fa1f522b1f2'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:05.853549 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-810c2a60-ec25-46b8-aef3-8fa1f522b1f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1e76c3fe4d4497a806ab9f0e36aede0', 'request_id': 'req-810c2a60-ec25-46b8-aef3-8fa1f522b1f2'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:05.859892 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-810c2a60-ec25-46b8-aef3-8fa1f522b1f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1e76c3fe4d4497a806ab9f0e36aede0', 'request_id': 'req-810c2a60-ec25-46b8-aef3-8fa1f522b1f2'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:05.871065 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-810c2a60-ec25-46b8-aef3-8fa1f522b1f2 admin admin] The token cache is being invalidate because project c1e76c3fe4d4497a806ab9f0e36aede0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:15:05.871378 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-810c2a60-ec25-46b8-aef3-8fa1f522b1f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:05.872052 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-810c2a60-ec25-46b8-aef3-8fa1f522b1f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:05.874084 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1297/2615] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:15:05 2022] DELETE /identity/v3/projects/c1e76c3fe4d4497a806ab9f0e36aede0 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:15:05.878096 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810c2a60-ec25-46b8-aef3-8fa1f522b1f2 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:05.879603 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-810c2a60-ec25-46b8-aef3-8fa1f522b1f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:05.894238 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1319/2616] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:15:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:15:05.904053 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec5545-c4e0-4c26-905c-7ed2b3cff38a None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:05.905407 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-08ec5545-c4e0-4c26-905c-7ed2b3cff38a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:05.945101 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810c2a60-ec25-46b8-aef3-8fa1f522b1f2 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:05.949735 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd31953-cd98-42ff-b6ad-1c17f4437d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:05.950592 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd31953-cd98-42ff-b6ad-1c17f4437d5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:05.950876 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd31953-cd98-42ff-b6ad-1c17f4437d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:05.951156 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd31953-cd98-42ff-b6ad-1c17f4437d5b admin admin] PATH_INFO: `/v3/users/a37e71346757447a8b68b6d5e6c0f3ba` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:05.952439 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd31953-cd98-42ff-b6ad-1c17f4437d5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a37e71346757447a8b68b6d5e6c0f3ba)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:05.954346 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd31953-cd98-42ff-b6ad-1c17f4437d5b admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:05.961759 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec5545-c4e0-4c26-905c-7ed2b3cff38a None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:05.965481 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52932663-6512-40b2-a759-d40bdf945802 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:05.966123 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52932663-6512-40b2-a759-d40bdf945802 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:05.966415 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52932663-6512-40b2-a759-d40bdf945802 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:05.966693 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52932663-6512-40b2-a759-d40bdf945802 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:05.976884 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52932663-6512-40b2-a759-d40bdf945802 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:05.978676 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52932663-6512-40b2-a759-d40bdf945802 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:05.984425 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:05.990038 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1320/2617] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:15:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 22:15:06.013980 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-efd31953-cd98-42ff-b6ad-1c17f4437d5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a37e71346757447a8b68b6d5e6c0f3ba'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:06.014651 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-efd31953-cd98-42ff-b6ad-1c17f4437d5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a37e71346757447a8b68b6d5e6c0f3ba'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:06.022871 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-efd31953-cd98-42ff-b6ad-1c17f4437d5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a37e71346757447a8b68b6d5e6c0f3ba'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:06.035851 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-efd31953-cd98-42ff-b6ad-1c17f4437d5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a37e71346757447a8b68b6d5e6c0f3ba'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:06.038023 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:06.044568 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-efd31953-cd98-42ff-b6ad-1c17f4437d5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a37e71346757447a8b68b6d5e6c0f3ba'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:06.053118 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1298/2618] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:15:05 2022] DELETE /identity/v3/users/a37e71346757447a8b68b6d5e6c0f3ba => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:15:06.317631 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52932663-6512-40b2-a759-d40bdf945802 service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:06.319100 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-52932663-6512-40b2-a759-d40bdf945802 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:06.332770 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52932663-6512-40b2-a759-d40bdf945802 service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:06.380869 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd31953-cd98-42ff-b6ad-1c17f4437d5b admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:06.382443 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-efd31953-cd98-42ff-b6ad-1c17f4437d5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:06.403095 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3ccfd4-988f-4f8b-82c8-c6c7062209c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:06.404409 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ccfd4-988f-4f8b-82c8-c6c7062209c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:06.404962 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ccfd4-988f-4f8b-82c8-c6c7062209c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:06.405817 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ccfd4-988f-4f8b-82c8-c6c7062209c7 admin admin] PATH_INFO: `/v3/projects/8f5e13ecb54c4181aff9244ada27e46e` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:06.407719 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3ccfd4-988f-4f8b-82c8-c6c7062209c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f5e13ecb54c4181aff9244ada27e46e)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:06.410101 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3ccfd4-988f-4f8b-82c8-c6c7062209c7 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:06.457136 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd31953-cd98-42ff-b6ad-1c17f4437d5b admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:06.457553 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8e3ccfd4-988f-4f8b-82c8-c6c7062209c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f5e13ecb54c4181aff9244ada27e46e', 'request_id': 'req-8e3ccfd4-988f-4f8b-82c8-c6c7062209c7'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:06.465157 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8e3ccfd4-988f-4f8b-82c8-c6c7062209c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f5e13ecb54c4181aff9244ada27e46e', 'request_id': 'req-8e3ccfd4-988f-4f8b-82c8-c6c7062209c7'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:06.471162 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8e3ccfd4-988f-4f8b-82c8-c6c7062209c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f5e13ecb54c4181aff9244ada27e46e', 'request_id': 'req-8e3ccfd4-988f-4f8b-82c8-c6c7062209c7'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:06.482413 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8e3ccfd4-988f-4f8b-82c8-c6c7062209c7 admin admin] The token cache is being invalidate because project 8f5e13ecb54c4181aff9244ada27e46e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:15:06.482705 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8e3ccfd4-988f-4f8b-82c8-c6c7062209c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:06.483370 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8e3ccfd4-988f-4f8b-82c8-c6c7062209c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:06.485139 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1321/2619] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:15:06 2022] DELETE /identity/v3/projects/8f5e13ecb54c4181aff9244ada27e46e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:15:06.489391 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3ccfd4-988f-4f8b-82c8-c6c7062209c7 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:06.490814 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-8e3ccfd4-988f-4f8b-82c8-c6c7062209c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:06.514655 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b50c0f-7494-41af-8686-3c67cef45730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:06.515554 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b50c0f-7494-41af-8686-3c67cef45730 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:06.515838 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b50c0f-7494-41af-8686-3c67cef45730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:06.516111 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b50c0f-7494-41af-8686-3c67cef45730 admin admin] PATH_INFO: `/v3/projects/34c8d06b29f14c24b82c8393b63c0846` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:06.517304 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b50c0f-7494-41af-8686-3c67cef45730 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34c8d06b29f14c24b82c8393b63c0846)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:06.519998 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b50c0f-7494-41af-8686-3c67cef45730 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:06.556631 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3ccfd4-988f-4f8b-82c8-c6c7062209c7 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:06.569122 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d4b50c0f-7494-41af-8686-3c67cef45730 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34c8d06b29f14c24b82c8393b63c0846', 'request_id': 'req-d4b50c0f-7494-41af-8686-3c67cef45730'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:06.576752 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d4b50c0f-7494-41af-8686-3c67cef45730 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34c8d06b29f14c24b82c8393b63c0846', 'request_id': 'req-d4b50c0f-7494-41af-8686-3c67cef45730'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:06.584440 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d4b50c0f-7494-41af-8686-3c67cef45730 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34c8d06b29f14c24b82c8393b63c0846', 'request_id': 'req-d4b50c0f-7494-41af-8686-3c67cef45730'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:06.594795 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d4b50c0f-7494-41af-8686-3c67cef45730 admin admin] The token cache is being invalidate because project 34c8d06b29f14c24b82c8393b63c0846 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:15:06.594943 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:06.595059 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d4b50c0f-7494-41af-8686-3c67cef45730 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:06.595660 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d4b50c0f-7494-41af-8686-3c67cef45730 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:06.597513 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1299/2620] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:15:06 2022] DELETE /identity/v3/projects/34c8d06b29f14c24b82c8393b63c0846 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:15:06.602210 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be9476-ec59-4213-9aed-a89a1f4102a0 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:06.602501 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be9476-ec59-4213-9aed-a89a1f4102a0 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:06.602782 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be9476-ec59-4213-9aed-a89a1f4102a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:06.626544 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-f0be9476-ec59-4213-9aed-a89a1f4102a0 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:06.627145 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c13de-2097-44ae-8d77-a9d11eef5d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:06.628588 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c13de-2097-44ae-8d77-a9d11eef5d93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:06.628874 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c13de-2097-44ae-8d77-a9d11eef5d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:06.629153 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c13de-2097-44ae-8d77-a9d11eef5d93 admin admin] PATH_INFO: `/v3/users/87c0384c2e414572a221399e08af4e2e` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:06.629313 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-f0be9476-ec59-4213-9aed-a89a1f4102a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:06.630514 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951c13de-2097-44ae-8d77-a9d11eef5d93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87c0384c2e414572a221399e08af4e2e)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:06.632843 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951c13de-2097-44ae-8d77-a9d11eef5d93 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:06.637294 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1300/2621] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:15:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:15:06.648913 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d984ea-8936-47b8-9bc3-48305fb51513 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:06.649225 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d984ea-8936-47b8-9bc3-48305fb51513 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:06.649523 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d984ea-8936-47b8-9bc3-48305fb51513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:06.668561 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-67d984ea-8936-47b8-9bc3-48305fb51513 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:06.669771 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-67d984ea-8936-47b8-9bc3-48305fb51513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:06.679313 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1301/2622] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:15:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:15:06.683274 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d984ea-8936-47b8-9bc3-48305fb51513 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:06.687013 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-951c13de-2097-44ae-8d77-a9d11eef5d93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87c0384c2e414572a221399e08af4e2e'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:06.693572 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d984ea-8936-47b8-9bc3-48305fb51513 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:06.694890 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-951c13de-2097-44ae-8d77-a9d11eef5d93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87c0384c2e414572a221399e08af4e2e'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:06.698914 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284995a6-8a72-4246-9dbe-3b525b4a9ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:06.699600 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284995a6-8a72-4246-9dbe-3b525b4a9ffd admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:06.699916 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284995a6-8a72-4246-9dbe-3b525b4a9ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:06.700237 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284995a6-8a72-4246-9dbe-3b525b4a9ffd admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:06.700992 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284995a6-8a72-4246-9dbe-3b525b4a9ffd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:15:06.701298 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284995a6-8a72-4246-9dbe-3b525b4a9ffd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:06.702884 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284995a6-8a72-4246-9dbe-3b525b4a9ffd admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:06.707515 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-951c13de-2097-44ae-8d77-a9d11eef5d93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87c0384c2e414572a221399e08af4e2e'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:06.708212 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-951c13de-2097-44ae-8d77-a9d11eef5d93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87c0384c2e414572a221399e08af4e2e'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:06.708822 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-951c13de-2097-44ae-8d77-a9d11eef5d93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87c0384c2e414572a221399e08af4e2e'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:06.718897 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1302/2623] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:15:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:15:06.719928 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284995a6-8a72-4246-9dbe-3b525b4a9ffd admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:06.721632 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-284995a6-8a72-4246-9dbe-3b525b4a9ffd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:06.724468 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1322/2624] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:15:06 2022] DELETE /identity/v3/users/87c0384c2e414572a221399e08af4e2e => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:15:06.729877 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:06.736497 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284995a6-8a72-4246-9dbe-3b525b4a9ffd admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:06.809706 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d12c212-ce5b-40f6-8400-171668dc8b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:06.810390 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d12c212-ce5b-40f6-8400-171668dc8b6b admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:06.810764 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d12c212-ce5b-40f6-8400-171668dc8b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:06.811055 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d12c212-ce5b-40f6-8400-171668dc8b6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:06.811785 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d12c212-ce5b-40f6-8400-171668dc8b6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:06.813911 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d12c212-ce5b-40f6-8400-171668dc8b6b admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:06.843260 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1303/2625] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:15:06 2022] POST /identity/v3/projects => generated 387 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:15:06.847838 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c13de-2097-44ae-8d77-a9d11eef5d93 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:06.859462 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c13de-2097-44ae-8d77-a9d11eef5d93 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:06.924739 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2373e5-592f-442b-bc00-ad0bea2e624c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:06.926101 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2373e5-592f-442b-bc00-ad0bea2e624c admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:06.926368 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2373e5-592f-442b-bc00-ad0bea2e624c admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:06.926651 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2373e5-592f-442b-bc00-ad0bea2e624c admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:06.927588 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2373e5-592f-442b-bc00-ad0bea2e624c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:06.929535 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2373e5-592f-442b-bc00-ad0bea2e624c admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:06.954220 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1323/2626] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:15:06 2022] POST /identity/v3/users => generated 294 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:15:06.955550 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d12c212-ce5b-40f6-8400-171668dc8b6b admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:06.967196 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d12c212-ce5b-40f6-8400-171668dc8b6b admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:06.973117 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad61a59-09d6-4b22-97ea-c86e1e6a5053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:06.973864 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad61a59-09d6-4b22-97ea-c86e1e6a5053 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:06.974214 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad61a59-09d6-4b22-97ea-c86e1e6a5053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:06.974565 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad61a59-09d6-4b22-97ea-c86e1e6a5053 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:06.975305 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad61a59-09d6-4b22-97ea-c86e1e6a5053 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:06.977449 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad61a59-09d6-4b22-97ea-c86e1e6a5053 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:06.995808 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1304/2627] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:15:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:15:06.997161 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2373e5-592f-442b-bc00-ad0bea2e624c admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:07.008432 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2373e5-592f-442b-bc00-ad0bea2e624c admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:07.013129 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d47f3d-245a-49c6-852a-a78b664efdb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:07.013803 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d47f3d-245a-49c6-852a-a78b664efdb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:07.014088 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d47f3d-245a-49c6-852a-a78b664efdb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:07.014393 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d47f3d-245a-49c6-852a-a78b664efdb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:07.015129 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d47f3d-245a-49c6-852a-a78b664efdb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:07.016774 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d47f3d-245a-49c6-852a-a78b664efdb7 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:07.027434 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1324/2628] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:15:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 22:15:07.031856 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad61a59-09d6-4b22-97ea-c86e1e6a5053 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:07.042515 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad61a59-09d6-4b22-97ea-c86e1e6a5053 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:07.048556 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7237ee94-1b7b-4fce-a3c1-36f742ad65bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:07.049639 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237ee94-1b7b-4fce-a3c1-36f742ad65bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:07.050718 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237ee94-1b7b-4fce-a3c1-36f742ad65bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:07.051075 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237ee94-1b7b-4fce-a3c1-36f742ad65bb admin admin] PATH_INFO: `/v3/projects/68e269f5712b4c4a9258ea0c0ea66428/users/87303f3ef19e47529cd2d957558c63bf/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:07.061627 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d47f3d-245a-49c6-852a-a78b664efdb7 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:07.063994 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-22d47f3d-245a-49c6-852a-a78b664efdb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:07.074249 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7237ee94-1b7b-4fce-a3c1-36f742ad65bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=68e269f5712b4c4a9258ea0c0ea66428, user_id=87303f3ef19e47529cd2d957558c63bf, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:07.078227 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7237ee94-1b7b-4fce-a3c1-36f742ad65bb admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:07.085240 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d47f3d-245a-49c6-852a-a78b664efdb7 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:07.092189 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb18f52f-d3a7-46ab-8051-1c56881be488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:07.093071 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18f52f-d3a7-46ab-8051-1c56881be488 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:07.093382 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18f52f-d3a7-46ab-8051-1c56881be488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:07.093620 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18f52f-d3a7-46ab-8051-1c56881be488 admin admin] PATH_INFO: `/v3/projects/9b201bf93b0a45d8b13dcaf79bcedba1` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:07.094844 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb18f52f-d3a7-46ab-8051-1c56881be488 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b201bf93b0a45d8b13dcaf79bcedba1)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:07.097335 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb18f52f-d3a7-46ab-8051-1c56881be488 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:07.097680 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1305/2629] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:15:07 2022] PUT /identity/v3/projects/68e269f5712b4c4a9258ea0c0ea66428/users/87303f3ef19e47529cd2d957558c63bf/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:15:07.108973 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1cd5f-62dd-48e4-8395-b3c8d2db9cc2 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:07.109514 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1cd5f-62dd-48e4-8395-b3c8d2db9cc2 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:07.109934 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1cd5f-62dd-48e4-8395-b3c8d2db9cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:07.137657 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=84,keystone:DELETE=12,keystone:INSERT=4 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:07.140846 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-7dc1cd5f-62dd-48e4-8395-b3c8d2db9cc2 None None] MFA Rules not processed for user `87303f3ef19e47529cd2d957558c63bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:07.142120 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-7dc1cd5f-62dd-48e4-8395-b3c8d2db9cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:07.154672 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-cb18f52f-d3a7-46ab-8051-1c56881be488 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b201bf93b0a45d8b13dcaf79bcedba1', 'request_id': 'req-cb18f52f-d3a7-46ab-8051-1c56881be488'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:07.163570 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-cb18f52f-d3a7-46ab-8051-1c56881be488 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b201bf93b0a45d8b13dcaf79bcedba1', 'request_id': 'req-cb18f52f-d3a7-46ab-8051-1c56881be488'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:07.170672 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-cb18f52f-d3a7-46ab-8051-1c56881be488 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b201bf93b0a45d8b13dcaf79bcedba1', 'request_id': 'req-cb18f52f-d3a7-46ab-8051-1c56881be488'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:07.183708 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-cb18f52f-d3a7-46ab-8051-1c56881be488 admin admin] The token cache is being invalidate because project 9b201bf93b0a45d8b13dcaf79bcedba1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:15:07.184000 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-cb18f52f-d3a7-46ab-8051-1c56881be488 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:07.184701 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-cb18f52f-d3a7-46ab-8051-1c56881be488 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:07.186522 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1325/2630] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:15:07 2022] DELETE /identity/v3/projects/9b201bf93b0a45d8b13dcaf79bcedba1 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:15:07.192172 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79609099-105e-4c9e-af8f-bc3894a87e43 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:07.192495 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79609099-105e-4c9e-af8f-bc3894a87e43 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:07.192864 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79609099-105e-4c9e-af8f-bc3894a87e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:07.215279 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1306/2631] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:15:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:15:07.217479 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-79609099-105e-4c9e-af8f-bc3894a87e43 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:07.218928 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-79609099-105e-4c9e-af8f-bc3894a87e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:07.219590 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1cd5f-62dd-48e4-8395-b3c8d2db9cc2 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:07.220882 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-7dc1cd5f-62dd-48e4-8395-b3c8d2db9cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:07.273781 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:07.292943 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1326/2632] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:15:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:15:07.297492 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1cd5f-62dd-48e4-8395-b3c8d2db9cc2 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:07.301196 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367750d-e4c4-4988-b48a-ce094c994012 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:07.301487 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367750d-e4c4-4988-b48a-ce094c994012 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:07.301772 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367750d-e4c4-4988-b48a-ce094c994012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:07.304521 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef46605-5991-473c-8e40-7afe7d76ee42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:07.305202 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef46605-5991-473c-8e40-7afe7d76ee42 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:07.305477 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef46605-5991-473c-8e40-7afe7d76ee42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:07.305746 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef46605-5991-473c-8e40-7afe7d76ee42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:07.306666 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef46605-5991-473c-8e40-7afe7d76ee42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:07.308624 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef46605-5991-473c-8e40-7afe7d76ee42 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:07.320655 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-9367750d-e4c4-4988-b48a-ce094c994012 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:07.321880 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-9367750d-e4c4-4988-b48a-ce094c994012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:07.334720 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1327/2633] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:15:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:15:07.340053 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367750d-e4c4-4988-b48a-ce094c994012 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:07.343811 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1307/2634] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:15:07 2022] POST /identity/v3/projects => generated 387 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:15:07.348432 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef46605-5991-473c-8e40-7afe7d76ee42 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:07.350420 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367750d-e4c4-4988-b48a-ce094c994012 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:07.414998 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997c8466-a1a4-40ac-a862-20b82a6aaa22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:07.415797 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c8466-a1a4-40ac-a862-20b82a6aaa22 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:07.416079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c8466-a1a4-40ac-a862-20b82a6aaa22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:07.416352 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c8466-a1a4-40ac-a862-20b82a6aaa22 admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:07.417081 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997c8466-a1a4-40ac-a862-20b82a6aaa22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:15:07.417387 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997c8466-a1a4-40ac-a862-20b82a6aaa22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:07.418991 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997c8466-a1a4-40ac-a862-20b82a6aaa22 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:07.422607 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef46605-5991-473c-8e40-7afe7d76ee42 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:07.429538 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f806c-6ecf-43ea-8ed8-681fc0b0bff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:07.430568 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f806c-6ecf-43ea-8ed8-681fc0b0bff5 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:07.430854 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f806c-6ecf-43ea-8ed8-681fc0b0bff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:07.431134 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f806c-6ecf-43ea-8ed8-681fc0b0bff5 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:07.431858 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299f806c-6ecf-43ea-8ed8-681fc0b0bff5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:07.437463 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1328/2635] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:15:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:15:07.437656 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299f806c-6ecf-43ea-8ed8-681fc0b0bff5 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:07.439073 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997c8466-a1a4-40ac-a862-20b82a6aaa22 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:07.453815 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997c8466-a1a4-40ac-a862-20b82a6aaa22 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:07.461101 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1308/2636] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:15:07 2022] POST /identity/v3/users => generated 300 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:15:07.461307 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac5d2a-590f-486c-a799-94f10190f43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:07.461307 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac5d2a-590f-486c-a799-94f10190f43e admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:07.461307 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac5d2a-590f-486c-a799-94f10190f43e admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:07.461307 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac5d2a-590f-486c-a799-94f10190f43e admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:07.461926 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ac5d2a-590f-486c-a799-94f10190f43e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:07.462134 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f806c-6ecf-43ea-8ed8-681fc0b0bff5 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:07.463863 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ac5d2a-590f-486c-a799-94f10190f43e admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:07.473584 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f806c-6ecf-43ea-8ed8-681fc0b0bff5 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:07.479579 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd95e2-2875-46c7-b5be-8ba8a856b1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:07.480252 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd95e2-2875-46c7-b5be-8ba8a856b1db admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:07.480534 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd95e2-2875-46c7-b5be-8ba8a856b1db admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:07.480815 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd95e2-2875-46c7-b5be-8ba8a856b1db admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:07.481555 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bd95e2-2875-46c7-b5be-8ba8a856b1db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:07.483174 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bd95e2-2875-46c7-b5be-8ba8a856b1db admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:07.494484 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1309/2637] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:15:07.494788 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1329/2638] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:15:07 2022] POST /identity/v3/projects => generated 389 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:15:07.499611 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd95e2-2875-46c7-b5be-8ba8a856b1db admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:07.502590 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac5d2a-590f-486c-a799-94f10190f43e admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:07.510277 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd95e2-2875-46c7-b5be-8ba8a856b1db admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:07.543720 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:07.574092 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d86b8e-a0df-499d-b02a-cad0c981728b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:07.574994 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d86b8e-a0df-499d-b02a-cad0c981728b admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:07.575279 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d86b8e-a0df-499d-b02a-cad0c981728b admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:07.575592 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d86b8e-a0df-499d-b02a-cad0c981728b admin admin] PATH_INFO: `/v3/projects/0cd1f26603d5440a91479b440a563aaf/users/ddab4183871c476990e76490d1e18f53/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:07.585977 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac5d2a-590f-486c-a799-94f10190f43e admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:07.592188 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eb3283-7ea0-4497-806e-6f938fc6c5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:07.592879 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eb3283-7ea0-4497-806e-6f938fc6c5b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:07.593169 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eb3283-7ea0-4497-806e-6f938fc6c5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:07.593450 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eb3283-7ea0-4497-806e-6f938fc6c5b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:07.593602 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d86b8e-a0df-499d-b02a-cad0c981728b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cd1f26603d5440a91479b440a563aaf, user_id=ddab4183871c476990e76490d1e18f53, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:07.594159 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eb3283-7ea0-4497-806e-6f938fc6c5b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:07.596079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eb3283-7ea0-4497-806e-6f938fc6c5b4 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:07.596303 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d86b8e-a0df-499d-b02a-cad0c981728b admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:07.614136 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1310/2639] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:15:07 2022] PUT /identity/v3/projects/0cd1f26603d5440a91479b440a563aaf/users/ddab4183871c476990e76490d1e18f53/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:15:07.619158 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d86b8e-a0df-499d-b02a-cad0c981728b admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:07.623208 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1330/2640] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:15:07 2022] POST /identity/v3/users => generated 295 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:15:07.624145 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eb3283-7ea0-4497-806e-6f938fc6c5b4 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:07.633786 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eb3283-7ea0-4497-806e-6f938fc6c5b4 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:07.654941 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:07.688678 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d86b8e-a0df-499d-b02a-cad0c981728b admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:07.692478 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d7dbb-5e05-4817-8550-cf2485399526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:07.693147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d7dbb-5e05-4817-8550-cf2485399526 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:07.693397 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe782fc-19b2-4e74-a16a-59570390fd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:07.693552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d7dbb-5e05-4817-8550-cf2485399526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:07.693687 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d7dbb-5e05-4817-8550-cf2485399526 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:07.694064 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe782fc-19b2-4e74-a16a-59570390fd33 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:07.694374 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe782fc-19b2-4e74-a16a-59570390fd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:07.694480 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343d7dbb-5e05-4817-8550-cf2485399526 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:07.694654 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe782fc-19b2-4e74-a16a-59570390fd33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:07.695426 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe782fc-19b2-4e74-a16a-59570390fd33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:07.696037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343d7dbb-5e05-4817-8550-cf2485399526 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:07.697000 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe782fc-19b2-4e74-a16a-59570390fd33 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:07.707611 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1331/2641] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 22:15:07.708204 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1311/2642] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 22:15:07.711167 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d7dbb-5e05-4817-8550-cf2485399526 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:07.711787 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe782fc-19b2-4e74-a16a-59570390fd33 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:07.721381 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d7dbb-5e05-4817-8550-cf2485399526 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:07.722816 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe782fc-19b2-4e74-a16a-59570390fd33 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:07.726778 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585640da-4c18-41f8-a16a-5a397bbf78c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:07.727477 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585640da-4c18-41f8-a16a-5a397bbf78c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:07.727610 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39b31fc-83b1-46e4-bcde-deacc8266ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:07.727766 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585640da-4c18-41f8-a16a-5a397bbf78c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:07.728047 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585640da-4c18-41f8-a16a-5a397bbf78c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:07.728680 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b31fc-83b1-46e4-bcde-deacc8266ec5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:07.728964 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b31fc-83b1-46e4-bcde-deacc8266ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:07.729080 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585640da-4c18-41f8-a16a-5a397bbf78c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:07.729241 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b31fc-83b1-46e4-bcde-deacc8266ec5 admin admin] PATH_INFO: `/v3/projects/0cd1f26603d5440a91479b440a563aaf/users/ddab4183871c476990e76490d1e18f53/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:07.730622 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585640da-4c18-41f8-a16a-5a397bbf78c1 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:07.731179 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39b31fc-83b1-46e4-bcde-deacc8266ec5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cd1f26603d5440a91479b440a563aaf, user_id=ddab4183871c476990e76490d1e18f53, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:07.735642 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39b31fc-83b1-46e4-bcde-deacc8266ec5 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:07.742773 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1332/2643] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:15:07.747663 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585640da-4c18-41f8-a16a-5a397bbf78c1 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:07.751086 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1312/2644] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:15:07 2022] PUT /identity/v3/projects/0cd1f26603d5440a91479b440a563aaf/users/ddab4183871c476990e76490d1e18f53/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:15:07.762300 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585640da-4c18-41f8-a16a-5a397bbf78c1 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:07.762736 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c798e29-113c-4967-9ed1-ed912ffd55b7 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:07.763030 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c798e29-113c-4967-9ed1-ed912ffd55b7 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:07.763313 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c798e29-113c-4967-9ed1-ed912ffd55b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:07.795256 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-7c798e29-113c-4967-9ed1-ed912ffd55b7 None None] MFA Rules not processed for user `ddab4183871c476990e76490d1e18f53`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:07.796513 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-7c798e29-113c-4967-9ed1-ed912ffd55b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:07.823611 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d69d83d-e3f6-4d8f-bb58-8440b741442e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:07.824521 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69d83d-e3f6-4d8f-bb58-8440b741442e admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:07.824834 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69d83d-e3f6-4d8f-bb58-8440b741442e admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:07.825117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69d83d-e3f6-4d8f-bb58-8440b741442e admin admin] PATH_INFO: `/v3/projects/4247a53b3d424287ad6441a0cae026c2/users/a5f414d4531448a684c41159c8c50e0d/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:07.841062 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d69d83d-e3f6-4d8f-bb58-8440b741442e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4247a53b3d424287ad6441a0cae026c2, user_id=a5f414d4531448a684c41159c8c50e0d, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:07.843787 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d69d83d-e3f6-4d8f-bb58-8440b741442e admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:07.857081 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1333/2645] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:15:07 2022] PUT /identity/v3/projects/4247a53b3d424287ad6441a0cae026c2/users/a5f414d4531448a684c41159c8c50e0d/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:15:07.867588 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1313/2646] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:15:07 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:15:07.867767 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36d5fb-4d77-4668-9ea5-ee87f9dda0d9 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:07.868060 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36d5fb-4d77-4668-9ea5-ee87f9dda0d9 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:07.868371 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36d5fb-4d77-4668-9ea5-ee87f9dda0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:07.878667 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c798e29-113c-4967-9ed1-ed912ffd55b7 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:07.880091 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-7c798e29-113c-4967-9ed1-ed912ffd55b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:07.896494 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-ef36d5fb-4d77-4668-9ea5-ee87f9dda0d9 None None] MFA Rules not processed for user `a5f414d4531448a684c41159c8c50e0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:07.897705 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-ef36d5fb-4d77-4668-9ea5-ee87f9dda0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:07.907007 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:07.938231 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c798e29-113c-4967-9ed1-ed912ffd55b7 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:07.964517 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1334/2647] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:15:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:15:07.968926 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef36d5fb-4d77-4668-9ea5-ee87f9dda0d9 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:07.988528 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdd120d-c554-44a4-9f27-fc0ddaed38e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:07.989280 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd120d-c554-44a4-9f27-fc0ddaed38e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:07.989645 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd120d-c554-44a4-9f27-fc0ddaed38e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:07.989923 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd120d-c554-44a4-9f27-fc0ddaed38e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:07.995239 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:08.033649 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef36d5fb-4d77-4668-9ea5-ee87f9dda0d9 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:08.038681 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a058f115-6712-4d9d-9de8-55582ca81bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:08.039362 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058f115-6712-4d9d-9de8-55582ca81bc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:08.039654 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058f115-6712-4d9d-9de8-55582ca81bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:08.039933 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058f115-6712-4d9d-9de8-55582ca81bc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:08.040626 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a058f115-6712-4d9d-9de8-55582ca81bc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:08.042550 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a058f115-6712-4d9d-9de8-55582ca81bc8 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:08.052418 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdd120d-c554-44a4-9f27-fc0ddaed38e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:08.053850 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdd120d-c554-44a4-9f27-fc0ddaed38e4 service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:08.067784 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1314/2648] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:15:07 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 22:15:08.071940 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1335/2649] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:15:07 2022] POST /identity/v3/projects => generated 389 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:15:08.076659 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdd120d-c554-44a4-9f27-fc0ddaed38e4 service cinder] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:08.088810 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdd120d-c554-44a4-9f27-fc0ddaed38e4 service cinder] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:08.152087 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246eb548-0744-4f8b-88d3-b72709732282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:08.152767 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246eb548-0744-4f8b-88d3-b72709732282 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:08.153052 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246eb548-0744-4f8b-88d3-b72709732282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:08.153331 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246eb548-0744-4f8b-88d3-b72709732282 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:08.154028 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246eb548-0744-4f8b-88d3-b72709732282 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:08.155967 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246eb548-0744-4f8b-88d3-b72709732282 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:08.181165 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1315/2650] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:15:08 2022] POST /identity/v3/users => generated 301 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:15:08.182303 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a058f115-6712-4d9d-9de8-55582ca81bc8 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:08.193986 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a058f115-6712-4d9d-9de8-55582ca81bc8 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:08.198626 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644cd57b-0e35-4252-ba62-40152ebf412b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:08.199287 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644cd57b-0e35-4252-ba62-40152ebf412b admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:08.199597 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644cd57b-0e35-4252-ba62-40152ebf412b admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:08.199872 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644cd57b-0e35-4252-ba62-40152ebf412b admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:08.200597 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644cd57b-0e35-4252-ba62-40152ebf412b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:08.202195 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644cd57b-0e35-4252-ba62-40152ebf412b admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:08.214672 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1336/2651] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:15:08.218868 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246eb548-0744-4f8b-88d3-b72709732282 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:08.229732 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246eb548-0744-4f8b-88d3-b72709732282 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:08.235184 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e0ffd-ad4a-403f-a42a-5a4a0b6c68ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:08.236201 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e0ffd-ad4a-403f-a42a-5a4a0b6c68ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:08.236483 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e0ffd-ad4a-403f-a42a-5a4a0b6c68ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:08.236756 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e0ffd-ad4a-403f-a42a-5a4a0b6c68ef admin admin] PATH_INFO: `/v3/projects/9002d905ba3d4a93a027d24c90a3240f/users/a6ea2b9587d34726954456a2f29356d0/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:08.253577 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5e0ffd-ad4a-403f-a42a-5a4a0b6c68ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=9002d905ba3d4a93a027d24c90a3240f, user_id=a6ea2b9587d34726954456a2f29356d0, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:08.256336 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5e0ffd-ad4a-403f-a42a-5a4a0b6c68ef admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:08.269819 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1316/2652] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:15:08 2022] PUT /identity/v3/projects/9002d905ba3d4a93a027d24c90a3240f/users/a6ea2b9587d34726954456a2f29356d0/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:15:08.274475 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644cd57b-0e35-4252-ba62-40152ebf412b admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:08.283859 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644cd57b-0e35-4252-ba62-40152ebf412b admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:08.346280 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f7340-0b38-41d1-a9cf-22deb74c5073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:08.347422 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f7340-0b38-41d1-a9cf-22deb74c5073 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:08.347705 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f7340-0b38-41d1-a9cf-22deb74c5073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:08.347984 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f7340-0b38-41d1-a9cf-22deb74c5073 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:08.348747 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740f7340-0b38-41d1-a9cf-22deb74c5073 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:08.350608 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740f7340-0b38-41d1-a9cf-22deb74c5073 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:08.364429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1337/2653] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:15:08.369673 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e0ffd-ad4a-403f-a42a-5a4a0b6c68ef admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:08.386583 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e0ffd-ad4a-403f-a42a-5a4a0b6c68ef admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:08.392865 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377e3093-c50d-4cfc-aa13-609a86b32fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:08.393803 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e3093-c50d-4cfc-aa13-609a86b32fc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:08.394091 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e3093-c50d-4cfc-aa13-609a86b32fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:08.394381 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e3093-c50d-4cfc-aa13-609a86b32fc2 admin admin] PATH_INFO: `/v3/projects/9002d905ba3d4a93a027d24c90a3240f/users/a6ea2b9587d34726954456a2f29356d0/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:08.396475 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377e3093-c50d-4cfc-aa13-609a86b32fc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9002d905ba3d4a93a027d24c90a3240f, user_id=a6ea2b9587d34726954456a2f29356d0, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:08.399140 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377e3093-c50d-4cfc-aa13-609a86b32fc2 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:08.410256 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1317/2654] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:15:08 2022] PUT /identity/v3/projects/9002d905ba3d4a93a027d24c90a3240f/users/a6ea2b9587d34726954456a2f29356d0/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:15:08.421569 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10d1b2-9829-43bf-8933-746fb009244c None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:08.421864 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10d1b2-9829-43bf-8933-746fb009244c None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:08.422146 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10d1b2-9829-43bf-8933-746fb009244c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:08.456114 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-9e10d1b2-9829-43bf-8933-746fb009244c None None] MFA Rules not processed for user `a6ea2b9587d34726954456a2f29356d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:08.457350 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-9e10d1b2-9829-43bf-8933-746fb009244c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:08.466684 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:08.535302 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1338/2655] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:15:08 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:15:08.542600 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377e3093-c50d-4cfc-aa13-609a86b32fc2 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:08.544463 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-377e3093-c50d-4cfc-aa13-609a86b32fc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:08.605759 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e10d1b2-9829-43bf-8933-746fb009244c None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:08.606178 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377e3093-c50d-4cfc-aa13-609a86b32fc2 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:08.610478 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62103f0-545d-40f1-bbe5-6db7b55a2ef4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:08.611471 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62103f0-545d-40f1-bbe5-6db7b55a2ef4 service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:08.611738 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62103f0-545d-40f1-bbe5-6db7b55a2ef4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:08.612018 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62103f0-545d-40f1-bbe5-6db7b55a2ef4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:08.622510 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62103f0-545d-40f1-bbe5-6db7b55a2ef4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:08.623966 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62103f0-545d-40f1-bbe5-6db7b55a2ef4 service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:08.629492 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:08.636600 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1318/2656] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:15:08 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:15:08.673677 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e10d1b2-9829-43bf-8933-746fb009244c None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:08.677159 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b50b3d4-8232-40b6-98ae-b68a97de23c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:08.678045 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b50b3d4-8232-40b6-98ae-b68a97de23c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:08.678317 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b50b3d4-8232-40b6-98ae-b68a97de23c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:08.678585 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b50b3d4-8232-40b6-98ae-b68a97de23c1 admin admin] PATH_INFO: `/v3/users/87303f3ef19e47529cd2d957558c63bf` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:08.679822 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b50b3d4-8232-40b6-98ae-b68a97de23c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87303f3ef19e47529cd2d957558c63bf)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:08.681788 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b50b3d4-8232-40b6-98ae-b68a97de23c1 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:08.736344 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8b50b3d4-8232-40b6-98ae-b68a97de23c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87303f3ef19e47529cd2d957558c63bf'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:08.744582 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8b50b3d4-8232-40b6-98ae-b68a97de23c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87303f3ef19e47529cd2d957558c63bf'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:08.760485 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8b50b3d4-8232-40b6-98ae-b68a97de23c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87303f3ef19e47529cd2d957558c63bf'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:08.761126 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8b50b3d4-8232-40b6-98ae-b68a97de23c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87303f3ef19e47529cd2d957558c63bf'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:08.761745 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8b50b3d4-8232-40b6-98ae-b68a97de23c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87303f3ef19e47529cd2d957558c63bf'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:08.772521 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62103f0-545d-40f1-bbe5-6db7b55a2ef4 service cinder] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:08.780183 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1339/2657] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:15:08 2022] DELETE /identity/v3/users/87303f3ef19e47529cd2d957558c63bf => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:15:08.783308 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62103f0-545d-40f1-bbe5-6db7b55a2ef4 service cinder] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:08.784579 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db08b5b-dc6b-4d43-b0a4-a6dc1e86c54d None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:08.784872 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db08b5b-dc6b-4d43-b0a4-a6dc1e86c54d None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:08.785156 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db08b5b-dc6b-4d43-b0a4-a6dc1e86c54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:08.803128 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-4db08b5b-dc6b-4d43-b0a4-a6dc1e86c54d None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:08.804373 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-4db08b5b-dc6b-4d43-b0a4-a6dc1e86c54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:08.825419 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:08.841848 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c934f852-6d91-4a0a-bfdc-de594c12905a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:08.842725 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934f852-6d91-4a0a-bfdc-de594c12905a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:08.843012 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934f852-6d91-4a0a-bfdc-de594c12905a admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:08.843559 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934f852-6d91-4a0a-bfdc-de594c12905a admin admin] PATH_INFO: `/v3/users/a5f414d4531448a684c41159c8c50e0d` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:08.844778 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c934f852-6d91-4a0a-bfdc-de594c12905a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5f414d4531448a684c41159c8c50e0d)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:08.846884 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c934f852-6d91-4a0a-bfdc-de594c12905a admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:08.867524 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1340/2658] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:15:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:15:08.878716 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db08b5b-dc6b-4d43-b0a4-a6dc1e86c54d None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:08.880487 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-4db08b5b-dc6b-4d43-b0a4-a6dc1e86c54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:08.896232 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c934f852-6d91-4a0a-bfdc-de594c12905a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5f414d4531448a684c41159c8c50e0d'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:08.896960 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c934f852-6d91-4a0a-bfdc-de594c12905a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5f414d4531448a684c41159c8c50e0d'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:08.904090 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c934f852-6d91-4a0a-bfdc-de594c12905a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5f414d4531448a684c41159c8c50e0d'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:08.916857 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c934f852-6d91-4a0a-bfdc-de594c12905a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5f414d4531448a684c41159c8c50e0d'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:08.923679 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c934f852-6d91-4a0a-bfdc-de594c12905a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5f414d4531448a684c41159c8c50e0d'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:08.930532 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1319/2659] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:15:08 2022] DELETE /identity/v3/users/a5f414d4531448a684c41159c8c50e0d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:15:08.934800 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb8f9a-6895-496c-915a-be09b92ed5bd None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:08.935157 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb8f9a-6895-496c-915a-be09b92ed5bd None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:08.935539 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb8f9a-6895-496c-915a-be09b92ed5bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:08.941326 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db08b5b-dc6b-4d43-b0a4-a6dc1e86c54d None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:08.952760 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-affb8f9a-6895-496c-915a-be09b92ed5bd None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:08.953984 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-affb8f9a-6895-496c-915a-be09b92ed5bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:08.993028 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca062ea-7c73-4e88-9a0d-2d74143d6239 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:08.993682 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca062ea-7c73-4e88-9a0d-2d74143d6239 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:08.993968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca062ea-7c73-4e88-9a0d-2d74143d6239 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:08.994244 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca062ea-7c73-4e88-9a0d-2d74143d6239 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:08.996060 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-fca062ea-7c73-4e88-9a0d-2d74143d6239 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:09.016704 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1320/2660] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:15:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:15:09.026266 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affb8f9a-6895-496c-915a-be09b92ed5bd None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:09.027608 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-affb8f9a-6895-496c-915a-be09b92ed5bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:09.039418 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affb8f9a-6895-496c-915a-be09b92ed5bd None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:09.043044 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb11d562-6e98-4acd-9205-478fe7f3c786 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:09.043727 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11d562-6e98-4acd-9205-478fe7f3c786 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:09.044011 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11d562-6e98-4acd-9205-478fe7f3c786 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:09.044288 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11d562-6e98-4acd-9205-478fe7f3c786 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:09.054044 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb11d562-6e98-4acd-9205-478fe7f3c786 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:09.055510 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb11d562-6e98-4acd-9205-478fe7f3c786 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:09.061230 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca062ea-7c73-4e88-9a0d-2d74143d6239 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:09.062666 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca062ea-7c73-4e88-9a0d-2d74143d6239 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:09.064959 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1321/2661] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:15:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 22:15:09.072721 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1341/2662] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:15:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 22:15:09.443511 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb11d562-6e98-4acd-9205-478fe7f3c786 service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:09.445287 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-bb11d562-6e98-4acd-9205-478fe7f3c786 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:09.457172 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:09.465110 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb11d562-6e98-4acd-9205-478fe7f3c786 service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:09.472079 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db56fd5-be03-4042-bc60-def8d75891ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:09.473215 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db56fd5-be03-4042-bc60-def8d75891ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:09.473788 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db56fd5-be03-4042-bc60-def8d75891ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:09.475585 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db56fd5-be03-4042-bc60-def8d75891ab admin admin] PATH_INFO: `/v3/projects/4247a53b3d424287ad6441a0cae026c2` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:09.477540 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca062ea-7c73-4e88-9a0d-2d74143d6239 service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:09.478003 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db56fd5-be03-4042-bc60-def8d75891ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=4247a53b3d424287ad6441a0cae026c2)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:09.479930 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-fca062ea-7c73-4e88-9a0d-2d74143d6239 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:09.481365 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db56fd5-be03-4042-bc60-def8d75891ab admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:09.509488 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca062ea-7c73-4e88-9a0d-2d74143d6239 service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:09.521064 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2a8fd9-382c-4254-b24b-9e19dd6a164c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:09.521947 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a8fd9-382c-4254-b24b-9e19dd6a164c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:09.522231 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a8fd9-382c-4254-b24b-9e19dd6a164c admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:09.522532 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a8fd9-382c-4254-b24b-9e19dd6a164c admin admin] PATH_INFO: `/v3/projects/68e269f5712b4c4a9258ea0c0ea66428` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:09.523795 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2a8fd9-382c-4254-b24b-9e19dd6a164c admin admin] RBAC: Authorizing `identity:delete_project(project_id=68e269f5712b4c4a9258ea0c0ea66428)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:09.525691 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2a8fd9-382c-4254-b24b-9e19dd6a164c admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:09.539782 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-6db56fd5-be03-4042-bc60-def8d75891ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4247a53b3d424287ad6441a0cae026c2', 'request_id': 'req-6db56fd5-be03-4042-bc60-def8d75891ab'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:09.547956 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-6db56fd5-be03-4042-bc60-def8d75891ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4247a53b3d424287ad6441a0cae026c2', 'request_id': 'req-6db56fd5-be03-4042-bc60-def8d75891ab'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:09.554983 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-6db56fd5-be03-4042-bc60-def8d75891ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4247a53b3d424287ad6441a0cae026c2', 'request_id': 'req-6db56fd5-be03-4042-bc60-def8d75891ab'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:09.560543 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:09.567091 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-6db56fd5-be03-4042-bc60-def8d75891ab admin admin] The token cache is being invalidate because project 4247a53b3d424287ad6441a0cae026c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:15:09.567530 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-6db56fd5-be03-4042-bc60-def8d75891ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:09.568175 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-6db56fd5-be03-4042-bc60-def8d75891ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:09.569711 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1322/2663] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:15:09 2022] DELETE /identity/v3/projects/4247a53b3d424287ad6441a0cae026c2 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:15:09.573598 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db56fd5-be03-4042-bc60-def8d75891ab admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:09.575081 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-6db56fd5-be03-4042-bc60-def8d75891ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:09.579835 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-da2a8fd9-382c-4254-b24b-9e19dd6a164c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68e269f5712b4c4a9258ea0c0ea66428', 'request_id': 'req-da2a8fd9-382c-4254-b24b-9e19dd6a164c'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:09.588404 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-da2a8fd9-382c-4254-b24b-9e19dd6a164c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68e269f5712b4c4a9258ea0c0ea66428', 'request_id': 'req-da2a8fd9-382c-4254-b24b-9e19dd6a164c'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:09.593847 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-da2a8fd9-382c-4254-b24b-9e19dd6a164c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68e269f5712b4c4a9258ea0c0ea66428', 'request_id': 'req-da2a8fd9-382c-4254-b24b-9e19dd6a164c'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:09.605331 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-da2a8fd9-382c-4254-b24b-9e19dd6a164c admin admin] The token cache is being invalidate because project 68e269f5712b4c4a9258ea0c0ea66428 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:15:09.605614 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-da2a8fd9-382c-4254-b24b-9e19dd6a164c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:09.606247 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-da2a8fd9-382c-4254-b24b-9e19dd6a164c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:09.607991 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1342/2664] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:15:09 2022] DELETE /identity/v3/projects/68e269f5712b4c4a9258ea0c0ea66428 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:15:09.611763 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2a8fd9-382c-4254-b24b-9e19dd6a164c admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:09.613161 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-da2a8fd9-382c-4254-b24b-9e19dd6a164c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:09.645900 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db56fd5-be03-4042-bc60-def8d75891ab admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:09.678211 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2a8fd9-382c-4254-b24b-9e19dd6a164c admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:09.683404 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a11402f-d9ea-46ce-92b7-2b0cf35153a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:09.684359 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11402f-d9ea-46ce-92b7-2b0cf35153a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:09.684651 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11402f-d9ea-46ce-92b7-2b0cf35153a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:09.684931 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11402f-d9ea-46ce-92b7-2b0cf35153a0 admin admin] PATH_INFO: `/v3/users/ddab4183871c476990e76490d1e18f53` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:09.686176 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a11402f-d9ea-46ce-92b7-2b0cf35153a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddab4183871c476990e76490d1e18f53)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:09.688124 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a11402f-d9ea-46ce-92b7-2b0cf35153a0 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:09.702369 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3068455b-ff37-430c-8f26-25ae8991dafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:09.703247 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3068455b-ff37-430c-8f26-25ae8991dafd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:09.703559 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3068455b-ff37-430c-8f26-25ae8991dafd admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:09.703836 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3068455b-ff37-430c-8f26-25ae8991dafd admin admin] PATH_INFO: `/v3/users/a6ea2b9587d34726954456a2f29356d0` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:09.705050 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3068455b-ff37-430c-8f26-25ae8991dafd admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6ea2b9587d34726954456a2f29356d0)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:09.706997 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3068455b-ff37-430c-8f26-25ae8991dafd admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:09.742563 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-0a11402f-d9ea-46ce-92b7-2b0cf35153a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddab4183871c476990e76490d1e18f53'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:09.752714 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-0a11402f-d9ea-46ce-92b7-2b0cf35153a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddab4183871c476990e76490d1e18f53'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:09.764867 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:09.769875 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-0a11402f-d9ea-46ce-92b7-2b0cf35153a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddab4183871c476990e76490d1e18f53'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:09.770638 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-0a11402f-d9ea-46ce-92b7-2b0cf35153a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddab4183871c476990e76490d1e18f53'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:09.771294 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-0a11402f-d9ea-46ce-92b7-2b0cf35153a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddab4183871c476990e76490d1e18f53'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:09.776881 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-3068455b-ff37-430c-8f26-25ae8991dafd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6ea2b9587d34726954456a2f29356d0'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:09.777774 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-3068455b-ff37-430c-8f26-25ae8991dafd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6ea2b9587d34726954456a2f29356d0'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:09.785334 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1343/2665] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:15:09 2022] DELETE /identity/v3/users/ddab4183871c476990e76490d1e18f53 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:15:09.785815 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-3068455b-ff37-430c-8f26-25ae8991dafd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6ea2b9587d34726954456a2f29356d0'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:09.798940 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-3068455b-ff37-430c-8f26-25ae8991dafd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6ea2b9587d34726954456a2f29356d0'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:09.806618 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-3068455b-ff37-430c-8f26-25ae8991dafd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6ea2b9587d34726954456a2f29356d0'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:09.812662 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1323/2666] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:15:09 2022] DELETE /identity/v3/users/a6ea2b9587d34726954456a2f29356d0 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:15:10.147675 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a11402f-d9ea-46ce-92b7-2b0cf35153a0 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:10.149318 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-0a11402f-d9ea-46ce-92b7-2b0cf35153a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:10.165482 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3068455b-ff37-430c-8f26-25ae8991dafd admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:10.166665 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-3068455b-ff37-430c-8f26-25ae8991dafd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:10.219054 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:10.239676 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3068455b-ff37-430c-8f26-25ae8991dafd admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:10.244220 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ff422-5d16-4fde-981d-4ff4889408d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:10.245083 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ff422-5d16-4fde-981d-4ff4889408d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:10.245369 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ff422-5d16-4fde-981d-4ff4889408d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:10.245643 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ff422-5d16-4fde-981d-4ff4889408d5 admin admin] PATH_INFO: `/v3/projects/9002d905ba3d4a93a027d24c90a3240f` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:10.246832 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924ff422-5d16-4fde-981d-4ff4889408d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9002d905ba3d4a93a027d24c90a3240f)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:10.247819 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a11402f-d9ea-46ce-92b7-2b0cf35153a0 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:10.248842 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924ff422-5d16-4fde-981d-4ff4889408d5 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:10.252481 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9bb4a8-9829-45eb-aa9e-259be727471d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:10.253381 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9bb4a8-9829-45eb-aa9e-259be727471d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:10.253671 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9bb4a8-9829-45eb-aa9e-259be727471d admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:10.253942 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9bb4a8-9829-45eb-aa9e-259be727471d admin admin] PATH_INFO: `/v3/projects/0cd1f26603d5440a91479b440a563aaf` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:10.255138 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9bb4a8-9829-45eb-aa9e-259be727471d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cd1f26603d5440a91479b440a563aaf)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:10.257071 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9bb4a8-9829-45eb-aa9e-259be727471d admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:10.305573 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-924ff422-5d16-4fde-981d-4ff4889408d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9002d905ba3d4a93a027d24c90a3240f', 'request_id': 'req-924ff422-5d16-4fde-981d-4ff4889408d5'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:10.313200 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-924ff422-5d16-4fde-981d-4ff4889408d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9002d905ba3d4a93a027d24c90a3240f', 'request_id': 'req-924ff422-5d16-4fde-981d-4ff4889408d5'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:10.313384 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-7d9bb4a8-9829-45eb-aa9e-259be727471d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cd1f26603d5440a91479b440a563aaf', 'request_id': 'req-7d9bb4a8-9829-45eb-aa9e-259be727471d'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:10.320503 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-924ff422-5d16-4fde-981d-4ff4889408d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9002d905ba3d4a93a027d24c90a3240f', 'request_id': 'req-924ff422-5d16-4fde-981d-4ff4889408d5'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:10.320720 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-7d9bb4a8-9829-45eb-aa9e-259be727471d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cd1f26603d5440a91479b440a563aaf', 'request_id': 'req-7d9bb4a8-9829-45eb-aa9e-259be727471d'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:10.326340 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-7d9bb4a8-9829-45eb-aa9e-259be727471d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cd1f26603d5440a91479b440a563aaf', 'request_id': 'req-7d9bb4a8-9829-45eb-aa9e-259be727471d'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:10.331916 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-924ff422-5d16-4fde-981d-4ff4889408d5 admin admin] The token cache is being invalidate because project 9002d905ba3d4a93a027d24c90a3240f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:15:10.332282 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-924ff422-5d16-4fde-981d-4ff4889408d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:10.332936 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-924ff422-5d16-4fde-981d-4ff4889408d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:10.334574 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1324/2667] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:15:10 2022] DELETE /identity/v3/projects/9002d905ba3d4a93a027d24c90a3240f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:15:10.338110 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-7d9bb4a8-9829-45eb-aa9e-259be727471d admin admin] The token cache is being invalidate because project 0cd1f26603d5440a91479b440a563aaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:15:10.338398 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-7d9bb4a8-9829-45eb-aa9e-259be727471d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:10.339041 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-7d9bb4a8-9829-45eb-aa9e-259be727471d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:10.339303 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513819ad-bcd1-4f15-a247-b438cc12c454 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:10.339617 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513819ad-bcd1-4f15-a247-b438cc12c454 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:10.339901 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513819ad-bcd1-4f15-a247-b438cc12c454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:10.340845 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1344/2668] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:15:10 2022] DELETE /identity/v3/projects/0cd1f26603d5440a91479b440a563aaf => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:15:10.345217 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4ff8e-8ede-4da3-9906-bc0e3bd18d8d None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:10.345505 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4ff8e-8ede-4da3-9906-bc0e3bd18d8d None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:10.345783 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4ff8e-8ede-4da3-9906-bc0e3bd18d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:10.361322 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-513819ad-bcd1-4f15-a247-b438cc12c454 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:10.363144 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-513819ad-bcd1-4f15-a247-b438cc12c454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:10.364514 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-c4c4ff8e-8ede-4da3-9906-bc0e3bd18d8d None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:10.365688 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c4c4ff8e-8ede-4da3-9906-bc0e3bd18d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:10.403889 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:10.425759 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1325/2669] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:15:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:15:10.428705 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1345/2670] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:15:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:15:10.436308 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70257c1e-917d-4b95-a047-41dc5ae27396 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:10.436447 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e50b8-2091-49ee-8095-47a1964015e4 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:10.436599 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70257c1e-917d-4b95-a047-41dc5ae27396 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:10.436705 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e50b8-2091-49ee-8095-47a1964015e4 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:10.436878 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70257c1e-917d-4b95-a047-41dc5ae27396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:10.436982 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e50b8-2091-49ee-8095-47a1964015e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:10.455785 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-975e50b8-2091-49ee-8095-47a1964015e4 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:10.457501 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-975e50b8-2091-49ee-8095-47a1964015e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:10.458206 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-70257c1e-917d-4b95-a047-41dc5ae27396 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:10.459451 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-70257c1e-917d-4b95-a047-41dc5ae27396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:10.465643 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1326/2671] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:15:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:15:10.467956 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1346/2672] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:15:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:15:10.469716 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975e50b8-2091-49ee-8095-47a1964015e4 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:10.471712 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70257c1e-917d-4b95-a047-41dc5ae27396 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:10.479694 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975e50b8-2091-49ee-8095-47a1964015e4 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:10.481745 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70257c1e-917d-4b95-a047-41dc5ae27396 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:10.484339 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac065484-9d05-401e-a770-e705f5e9669a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:10.484990 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac065484-9d05-401e-a770-e705f5e9669a admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:10.485275 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac065484-9d05-401e-a770-e705f5e9669a admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:10.485555 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac065484-9d05-401e-a770-e705f5e9669a admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:10.486275 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac065484-9d05-401e-a770-e705f5e9669a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:15:10.486579 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac065484-9d05-401e-a770-e705f5e9669a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:10.486681 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca979836-41c4-4716-91ca-b21844f96091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:10.487418 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca979836-41c4-4716-91ca-b21844f96091 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:10.487711 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca979836-41c4-4716-91ca-b21844f96091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:10.487976 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca979836-41c4-4716-91ca-b21844f96091 admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:10.488409 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac065484-9d05-401e-a770-e705f5e9669a admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:10.488707 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca979836-41c4-4716-91ca-b21844f96091 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:15:10.489016 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca979836-41c4-4716-91ca-b21844f96091 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:10.491034 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca979836-41c4-4716-91ca-b21844f96091 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:10.500160 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1327/2673] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:15:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 22:15:10.505442 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1347/2674] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:15:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 22:15:10.505588 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac065484-9d05-401e-a770-e705f5e9669a admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:10.506341 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca979836-41c4-4716-91ca-b21844f96091 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:10.514797 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac065484-9d05-401e-a770-e705f5e9669a admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:10.515908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca979836-41c4-4716-91ca-b21844f96091 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:10.520100 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948a7382-7d87-4aef-8cb3-bfb6c7baaf16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:10.520858 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a7382-7d87-4aef-8cb3-bfb6c7baaf16 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:10.521147 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a7382-7d87-4aef-8cb3-bfb6c7baaf16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:10.521251 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8afbef1-ecd8-4141-849f-c119ec1419d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:10.521428 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a7382-7d87-4aef-8cb3-bfb6c7baaf16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:10.521816 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8afbef1-ecd8-4141-849f-c119ec1419d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:10.522104 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8afbef1-ecd8-4141-849f-c119ec1419d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:10.522221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948a7382-7d87-4aef-8cb3-bfb6c7baaf16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:10.522386 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8afbef1-ecd8-4141-849f-c119ec1419d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:10.523066 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8afbef1-ecd8-4141-849f-c119ec1419d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:10.524112 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948a7382-7d87-4aef-8cb3-bfb6c7baaf16 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:10.525022 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8afbef1-ecd8-4141-849f-c119ec1419d9 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:10.555309 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1348/2675] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:15:10 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:15:10.555570 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1328/2676] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:15:10 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:15:10.556769 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948a7382-7d87-4aef-8cb3-bfb6c7baaf16 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:10.556913 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8afbef1-ecd8-4141-849f-c119ec1419d9 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:10.583870 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:10.621724 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8afbef1-ecd8-4141-849f-c119ec1419d9 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:10.622592 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948a7382-7d87-4aef-8cb3-bfb6c7baaf16 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:10.626397 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4349979f-7235-4c4d-a5b0-2392c6c227f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:10.627060 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349979f-7235-4c4d-a5b0-2392c6c227f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:10.627363 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349979f-7235-4c4d-a5b0-2392c6c227f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:10.627492 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb5e33b-6825-47e5-b881-c29f29c35cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:10.627640 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349979f-7235-4c4d-a5b0-2392c6c227f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:10.628761 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb5e33b-6825-47e5-b881-c29f29c35cba admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:10.629069 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb5e33b-6825-47e5-b881-c29f29c35cba admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:10.629174 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4349979f-7235-4c4d-a5b0-2392c6c227f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:10.629375 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb5e33b-6825-47e5-b881-c29f29c35cba admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:10.630066 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb5e33b-6825-47e5-b881-c29f29c35cba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:10.631002 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4349979f-7235-4c4d-a5b0-2392c6c227f3 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:10.631995 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb5e33b-6825-47e5-b881-c29f29c35cba admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:10.654744 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1329/2678] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:15:10 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:15:10.654973 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1349/2678] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:15:10 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:15:10.655859 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb5e33b-6825-47e5-b881-c29f29c35cba admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:10.656000 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4349979f-7235-4c4d-a5b0-2392c6c227f3 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:10.666011 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4349979f-7235-4c4d-a5b0-2392c6c227f3 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:10.666176 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb5e33b-6825-47e5-b881-c29f29c35cba admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:10.670844 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bb22ed-ba6d-40ba-8ce5-8420d2644e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:10.670984 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256b5b73-72fc-4dc1-8b24-5e0ded78656a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:10.671535 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb22ed-ba6d-40ba-8ce5-8420d2644e67 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:10.671650 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b5b73-72fc-4dc1-8b24-5e0ded78656a admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:10.671821 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb22ed-ba6d-40ba-8ce5-8420d2644e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:10.671944 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b5b73-72fc-4dc1-8b24-5e0ded78656a admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:10.672097 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b5b73-72fc-4dc1-8b24-5e0ded78656a admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:10.672196 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb22ed-ba6d-40ba-8ce5-8420d2644e67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:10.672842 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bb22ed-ba6d-40ba-8ce5-8420d2644e67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:10.672961 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256b5b73-72fc-4dc1-8b24-5e0ded78656a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:10.674429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256b5b73-72fc-4dc1-8b24-5e0ded78656a admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:10.674555 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bb22ed-ba6d-40ba-8ce5-8420d2644e67 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:10.685114 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1330/2679] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 22:15:10.685375 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1350/2680] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:15:10.688834 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256b5b73-72fc-4dc1-8b24-5e0ded78656a admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:10.689059 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bb22ed-ba6d-40ba-8ce5-8420d2644e67 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:10.699217 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bb22ed-ba6d-40ba-8ce5-8420d2644e67 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:10.699420 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256b5b73-72fc-4dc1-8b24-5e0ded78656a admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:10.704003 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6e9521-e78e-45c2-8109-6c8f0d3103e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:10.704602 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdc261f-d53a-43ea-b501-0fc09c94745f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:10.705251 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdc261f-d53a-43ea-b501-0fc09c94745f admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:10.705370 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e9521-e78e-45c2-8109-6c8f0d3103e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:10.705536 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdc261f-d53a-43ea-b501-0fc09c94745f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:10.705667 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e9521-e78e-45c2-8109-6c8f0d3103e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:10.705813 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdc261f-d53a-43ea-b501-0fc09c94745f admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:10.705955 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e9521-e78e-45c2-8109-6c8f0d3103e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:10.706553 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6e9521-e78e-45c2-8109-6c8f0d3103e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:10.706670 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdc261f-d53a-43ea-b501-0fc09c94745f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:10.708171 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdc261f-d53a-43ea-b501-0fc09c94745f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:10.708306 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6e9521-e78e-45c2-8109-6c8f0d3103e7 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:10.721263 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1351/2681] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:15:10.721454 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1331/2682] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 22:15:10.722662 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdc261f-d53a-43ea-b501-0fc09c94745f admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:10.722801 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6e9521-e78e-45c2-8109-6c8f0d3103e7 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:10.732407 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6e9521-e78e-45c2-8109-6c8f0d3103e7 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:10.732826 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdc261f-d53a-43ea-b501-0fc09c94745f admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:10.736878 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be52e281-3f23-407b-9bef-b99639e0a33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:10.737960 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5df9bc-26b5-460a-abf3-5e24d7a40927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:10.738332 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be52e281-3f23-407b-9bef-b99639e0a33c admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:10.738608 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be52e281-3f23-407b-9bef-b99639e0a33c admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:10.738822 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5df9bc-26b5-460a-abf3-5e24d7a40927 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:10.738929 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be52e281-3f23-407b-9bef-b99639e0a33c admin admin] PATH_INFO: `/v3/projects/b21e9ae591604b779cbb7ad17f6b3fb5/users/ed8e3ca088c4452689969585b8ce6fd8/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:10.739111 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5df9bc-26b5-460a-abf3-5e24d7a40927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:10.739485 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5df9bc-26b5-460a-abf3-5e24d7a40927 admin admin] PATH_INFO: `/v3/projects/cdb7b4f88370455baff2e9ee434700fe/users/c32abf74f4de4519a8b7552f70ed4192/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:10.754688 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be52e281-3f23-407b-9bef-b99639e0a33c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b21e9ae591604b779cbb7ad17f6b3fb5, user_id=ed8e3ca088c4452689969585b8ce6fd8, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:10.757145 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5df9bc-26b5-460a-abf3-5e24d7a40927 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdb7b4f88370455baff2e9ee434700fe, user_id=c32abf74f4de4519a8b7552f70ed4192, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:10.757363 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be52e281-3f23-407b-9bef-b99639e0a33c admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:10.760941 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5df9bc-26b5-460a-abf3-5e24d7a40927 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:10.773339 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1332/2683] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:15:10 2022] PUT /identity/v3/projects/b21e9ae591604b779cbb7ad17f6b3fb5/users/ed8e3ca088c4452689969585b8ce6fd8/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:15:10.778046 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1352/2684] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:15:10 2022] PUT /identity/v3/projects/cdb7b4f88370455baff2e9ee434700fe/users/c32abf74f4de4519a8b7552f70ed4192/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:15:10.784881 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2635fb-33a4-422d-a7a4-403b9031a6f9 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:10.785172 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2635fb-33a4-422d-a7a4-403b9031a6f9 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:10.785450 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2635fb-33a4-422d-a7a4-403b9031a6f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:10.788713 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a723ac-2b35-44d3-b099-1b58fb8a8d02 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:10.789150 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a723ac-2b35-44d3-b099-1b58fb8a8d02 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:10.789438 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a723ac-2b35-44d3-b099-1b58fb8a8d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:10.812048 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-af2635fb-33a4-422d-a7a4-403b9031a6f9 None None] MFA Rules not processed for user `ed8e3ca088c4452689969585b8ce6fd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:10.813177 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-af2635fb-33a4-422d-a7a4-403b9031a6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:10.816484 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-a3a723ac-2b35-44d3-b099-1b58fb8a8d02 None None] MFA Rules not processed for user `c32abf74f4de4519a8b7552f70ed4192`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:10.817681 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a3a723ac-2b35-44d3-b099-1b58fb8a8d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:10.847592 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:10.880095 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1333/2685] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:15:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:15:10.884554 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2635fb-33a4-422d-a7a4-403b9031a6f9 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:10.885421 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1353/2686] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:15:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:15:10.889653 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a723ac-2b35-44d3-b099-1b58fb8a8d02 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:10.952747 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2635fb-33a4-422d-a7a4-403b9031a6f9 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:10.958501 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a786814d-0704-4e78-98e5-63e6a792d68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:10.959199 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a786814d-0704-4e78-98e5-63e6a792d68f admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:10.959533 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a786814d-0704-4e78-98e5-63e6a792d68f admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:10.959817 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a786814d-0704-4e78-98e5-63e6a792d68f admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:10.960529 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a786814d-0704-4e78-98e5-63e6a792d68f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:10.962505 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a786814d-0704-4e78-98e5-63e6a792d68f admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:10.993925 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1334/2687] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:15:10 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:15:10.995112 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a786814d-0704-4e78-98e5-63e6a792d68f admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:11.064409 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a786814d-0704-4e78-98e5-63e6a792d68f admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:11.070681 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1287f04-5698-407e-98c7-eb06b5ab5e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:11.071418 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1287f04-5698-407e-98c7-eb06b5ab5e1c admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:11.072273 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1287f04-5698-407e-98c7-eb06b5ab5e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:11.072273 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1287f04-5698-407e-98c7-eb06b5ab5e1c admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:11.072673 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1287f04-5698-407e-98c7-eb06b5ab5e1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:11.074963 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1287f04-5698-407e-98c7-eb06b5ab5e1c admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:11.089214 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:11.094169 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a723ac-2b35-44d3-b099-1b58fb8a8d02 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:11.095739 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1335/2688] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:15:10 2022] POST /identity/v3/users => generated 294 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:15:11.099657 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1287f04-5698-407e-98c7-eb06b5ab5e1c admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:11.100545 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41566e7-985a-4faf-be3f-1f6bd00946a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:11.101203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41566e7-985a-4faf-be3f-1f6bd00946a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:11.101484 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41566e7-985a-4faf-be3f-1f6bd00946a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:11.101758 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41566e7-985a-4faf-be3f-1f6bd00946a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:11.102437 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41566e7-985a-4faf-be3f-1f6bd00946a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:11.104606 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41566e7-985a-4faf-be3f-1f6bd00946a6 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:11.109408 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1287f04-5698-407e-98c7-eb06b5ab5e1c admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:11.114365 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e893a4b4-c869-4b88-8a3f-636df316165a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:11.115014 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893a4b4-c869-4b88-8a3f-636df316165a admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:11.115303 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893a4b4-c869-4b88-8a3f-636df316165a admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:11.115615 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893a4b4-c869-4b88-8a3f-636df316165a admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:11.116337 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e893a4b4-c869-4b88-8a3f-636df316165a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:11.117955 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e893a4b4-c869-4b88-8a3f-636df316165a admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:11.130258 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1336/2689] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:15:11.134443 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e893a4b4-c869-4b88-8a3f-636df316165a admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:11.138155 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1354/2690] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:15:10 2022] POST /identity/v3/projects => generated 379 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:15:11.139039 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41566e7-985a-4faf-be3f-1f6bd00946a6 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:11.144927 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e893a4b4-c869-4b88-8a3f-636df316165a admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:11.183880 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:11.203941 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41566e7-985a-4faf-be3f-1f6bd00946a6 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:11.205171 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2e4fb2-1ef6-48fc-95ab-5ba471ec718a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:11.206162 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e4fb2-1ef6-48fc-95ab-5ba471ec718a admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:11.206579 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e4fb2-1ef6-48fc-95ab-5ba471ec718a admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:11.206860 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e4fb2-1ef6-48fc-95ab-5ba471ec718a admin admin] PATH_INFO: `/v3/projects/e721ac34a87a41c38328e398e34eb16a/users/cf1c97847ab74097a33597708ab86aac/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:11.208722 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c2b7b5-7dc7-4b75-bc59-c7b05eaaf999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:11.209385 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2b7b5-7dc7-4b75-bc59-c7b05eaaf999 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:11.209669 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2b7b5-7dc7-4b75-bc59-c7b05eaaf999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:11.209945 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2b7b5-7dc7-4b75-bc59-c7b05eaaf999 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:11.210660 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c2b7b5-7dc7-4b75-bc59-c7b05eaaf999 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:11.212597 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c2b7b5-7dc7-4b75-bc59-c7b05eaaf999 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:11.221598 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2e4fb2-1ef6-48fc-95ab-5ba471ec718a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e721ac34a87a41c38328e398e34eb16a, user_id=cf1c97847ab74097a33597708ab86aac, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:11.224313 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2e4fb2-1ef6-48fc-95ab-5ba471ec718a admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:11.233345 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1355/2691] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:15:11 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:15:11.237115 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c2b7b5-7dc7-4b75-bc59-c7b05eaaf999 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:11.239100 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1337/2692] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:15:11 2022] PUT /identity/v3/projects/e721ac34a87a41c38328e398e34eb16a/users/cf1c97847ab74097a33597708ab86aac/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:15:11.242732 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2e4fb2-1ef6-48fc-95ab-5ba471ec718a admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:11.247244 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c2b7b5-7dc7-4b75-bc59-c7b05eaaf999 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:11.305308 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0488dbe6-581f-447c-b82e-e9f92043c9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:11.305981 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0488dbe6-581f-447c-b82e-e9f92043c9f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:11.306263 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0488dbe6-581f-447c-b82e-e9f92043c9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:11.306540 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0488dbe6-581f-447c-b82e-e9f92043c9f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:11.307263 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0488dbe6-581f-447c-b82e-e9f92043c9f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:11.308882 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0488dbe6-581f-447c-b82e-e9f92043c9f0 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:11.310041 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2e4fb2-1ef6-48fc-95ab-5ba471ec718a admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:11.315092 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c01f3d-aa1b-4b02-ac16-89eba9ec5394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:11.316264 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c01f3d-aa1b-4b02-ac16-89eba9ec5394 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:11.316662 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c01f3d-aa1b-4b02-ac16-89eba9ec5394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:11.316979 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c01f3d-aa1b-4b02-ac16-89eba9ec5394 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:11.317718 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c01f3d-aa1b-4b02-ac16-89eba9ec5394 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:11.319155 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1356/2693] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 22:15:11.322172 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c01f3d-aa1b-4b02-ac16-89eba9ec5394 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:11.322955 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0488dbe6-581f-447c-b82e-e9f92043c9f0 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:11.332341 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0488dbe6-581f-447c-b82e-e9f92043c9f0 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:11.335865 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1338/2694] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 22:15:11.336845 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c01f3d-aa1b-4b02-ac16-89eba9ec5394 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:11.338331 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb24b9e9-24e0-49aa-8005-e6aa85162470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:11.338990 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24b9e9-24e0-49aa-8005-e6aa85162470 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:11.339270 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24b9e9-24e0-49aa-8005-e6aa85162470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:11.339577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24b9e9-24e0-49aa-8005-e6aa85162470 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:11.340304 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb24b9e9-24e0-49aa-8005-e6aa85162470 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:11.341890 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb24b9e9-24e0-49aa-8005-e6aa85162470 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:11.346898 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c01f3d-aa1b-4b02-ac16-89eba9ec5394 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:11.351584 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74899c8-3685-4fbb-8d5e-f7f188155e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:11.352465 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74899c8-3685-4fbb-8d5e-f7f188155e4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:11.352614 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1357/2695] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 22:15:11.352766 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74899c8-3685-4fbb-8d5e-f7f188155e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:11.353024 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74899c8-3685-4fbb-8d5e-f7f188155e4c admin admin] PATH_INFO: `/v3/projects/e721ac34a87a41c38328e398e34eb16a/users/cf1c97847ab74097a33597708ab86aac/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:11.355441 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74899c8-3685-4fbb-8d5e-f7f188155e4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e721ac34a87a41c38328e398e34eb16a, user_id=cf1c97847ab74097a33597708ab86aac, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:11.356826 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb24b9e9-24e0-49aa-8005-e6aa85162470 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:11.358321 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74899c8-3685-4fbb-8d5e-f7f188155e4c admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:11.367044 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb24b9e9-24e0-49aa-8005-e6aa85162470 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:11.369748 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1339/2696] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:15:11 2022] PUT /identity/v3/projects/e721ac34a87a41c38328e398e34eb16a/users/cf1c97847ab74097a33597708ab86aac/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:15:11.421115 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:11.429759 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0ed44c-6fb6-4f12-b85c-426d0ec3985c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:11.430807 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0ed44c-6fb6-4f12-b85c-426d0ec3985c admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:11.431346 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0ed44c-6fb6-4f12-b85c-426d0ec3985c admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:11.431726 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0ed44c-6fb6-4f12-b85c-426d0ec3985c admin admin] PATH_INFO: `/v3/projects/8580534b941b4617b38d4d3d41f353e4/users/194bd33031e2401f9d5862bcd38257bb/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:11.437533 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb290c-e6cc-44e6-9314-3d78ed7aa4e5 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:11.437825 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb290c-e6cc-44e6-9314-3d78ed7aa4e5 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:11.438104 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb290c-e6cc-44e6-9314-3d78ed7aa4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:11.446872 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0ed44c-6fb6-4f12-b85c-426d0ec3985c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8580534b941b4617b38d4d3d41f353e4, user_id=194bd33031e2401f9d5862bcd38257bb, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:11.449586 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0ed44c-6fb6-4f12-b85c-426d0ec3985c admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:11.462385 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1358/2697] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:15:11 2022] PUT /identity/v3/projects/8580534b941b4617b38d4d3d41f353e4/users/194bd33031e2401f9d5862bcd38257bb/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:15:11.466181 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-7ceb290c-e6cc-44e6-9314-3d78ed7aa4e5 None None] MFA Rules not processed for user `cf1c97847ab74097a33597708ab86aac`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:11.467424 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-7ceb290c-e6cc-44e6-9314-3d78ed7aa4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:11.528626 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c346fef-a8df-4e5d-bd3a-6320540c14c2 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:11.529177 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c346fef-a8df-4e5d-bd3a-6320540c14c2 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:11.529177 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c346fef-a8df-4e5d-bd3a-6320540c14c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:11.553868 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1340/2698] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:15:11 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:15:11.556712 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-4c346fef-a8df-4e5d-bd3a-6320540c14c2 None None] MFA Rules not processed for user `194bd33031e2401f9d5862bcd38257bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:11.558292 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-4c346fef-a8df-4e5d-bd3a-6320540c14c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:11.566336 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceb290c-e6cc-44e6-9314-3d78ed7aa4e5 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:11.567928 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-7ceb290c-e6cc-44e6-9314-3d78ed7aa4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:11.603108 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:11.629039 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1359/2699] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:15:11 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:15:11.629247 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceb290c-e6cc-44e6-9314-3d78ed7aa4e5 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:11.632795 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff12f04b-aa39-44e3-bff2-4486a1730c4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:11.633455 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12f04b-aa39-44e3-bff2-4486a1730c4a service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:11.633620 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c346fef-a8df-4e5d-bd3a-6320540c14c2 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:11.633735 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12f04b-aa39-44e3-bff2-4486a1730c4a service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:11.634011 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12f04b-aa39-44e3-bff2-4486a1730c4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:11.693880 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff12f04b-aa39-44e3-bff2-4486a1730c4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:11.695315 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff12f04b-aa39-44e3-bff2-4486a1730c4a service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:11.697701 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c346fef-a8df-4e5d-bd3a-6320540c14c2 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:11.702556 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a38fa-56e9-4280-8625-30f9952867e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:11.703238 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a38fa-56e9-4280-8625-30f9952867e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:11.703550 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a38fa-56e9-4280-8625-30f9952867e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:11.703827 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a38fa-56e9-4280-8625-30f9952867e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:11.704319 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1341/2700] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:15:11 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 22:15:11.704517 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48a38fa-56e9-4280-8625-30f9952867e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:11.707192 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48a38fa-56e9-4280-8625-30f9952867e0 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:11.733770 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1360/2701] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:15:11 2022] POST /identity/v3/projects => generated 377 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:15:11.738195 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff12f04b-aa39-44e3-bff2-4486a1730c4a service cinder] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:11.749636 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff12f04b-aa39-44e3-bff2-4486a1730c4a service cinder] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:11.820031 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddcb21f-5c8d-48dc-a4a1-e5e854e9f471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:11.820612 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddcb21f-5c8d-48dc-a4a1-e5e854e9f471 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:11.820891 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddcb21f-5c8d-48dc-a4a1-e5e854e9f471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:11.821162 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddcb21f-5c8d-48dc-a4a1-e5e854e9f471 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:11.821845 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddcb21f-5c8d-48dc-a4a1-e5e854e9f471 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:11.823854 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddcb21f-5c8d-48dc-a4a1-e5e854e9f471 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:11.845543 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1342/2702] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:15:11 2022] POST /identity/v3/users => generated 295 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:15:11.849793 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a38fa-56e9-4280-8625-30f9952867e0 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:11.861395 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a38fa-56e9-4280-8625-30f9952867e0 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:11.867448 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810b5191-f7d3-412a-a431-ce3241643f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:11.868116 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b5191-f7d3-412a-a431-ce3241643f34 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:11.868529 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b5191-f7d3-412a-a431-ce3241643f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:11.868806 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b5191-f7d3-412a-a431-ce3241643f34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:11.869539 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810b5191-f7d3-412a-a431-ce3241643f34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:11.871359 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810b5191-f7d3-412a-a431-ce3241643f34 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:11.882794 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1361/2703] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 22:15:11.887179 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddcb21f-5c8d-48dc-a4a1-e5e854e9f471 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:11.898360 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddcb21f-5c8d-48dc-a4a1-e5e854e9f471 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:11.903048 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9536f8-de10-439f-85a9-e683b2500675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:11.903958 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9536f8-de10-439f-85a9-e683b2500675 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:11.904247 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9536f8-de10-439f-85a9-e683b2500675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:11.904531 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9536f8-de10-439f-85a9-e683b2500675 admin admin] PATH_INFO: `/v3/projects/5ba2499f30044560b6eeb26af6019a51/users/57a98c89932145f38c22a86bf9d34766/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:11.921222 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9536f8-de10-439f-85a9-e683b2500675 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ba2499f30044560b6eeb26af6019a51, user_id=57a98c89932145f38c22a86bf9d34766, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:11.923968 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9536f8-de10-439f-85a9-e683b2500675 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:11.938251 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1343/2704] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:15:11 2022] PUT /identity/v3/projects/5ba2499f30044560b6eeb26af6019a51/users/57a98c89932145f38c22a86bf9d34766/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:15:11.943212 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810b5191-f7d3-412a-a431-ce3241643f34 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:11.956648 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810b5191-f7d3-412a-a431-ce3241643f34 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:12.014329 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:12.031677 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18d0b8e-f939-4847-a8fe-20f9cc5d1600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:12.032348 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d0b8e-f939-4847-a8fe-20f9cc5d1600 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:12.032632 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d0b8e-f939-4847-a8fe-20f9cc5d1600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:12.032910 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d0b8e-f939-4847-a8fe-20f9cc5d1600 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:12.033638 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18d0b8e-f939-4847-a8fe-20f9cc5d1600 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:12.035659 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18d0b8e-f939-4847-a8fe-20f9cc5d1600 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:12.045614 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1362/2705] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 22:15:12.050763 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9536f8-de10-439f-85a9-e683b2500675 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:12.061456 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9536f8-de10-439f-85a9-e683b2500675 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:12.066760 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e195378f-6c59-4c09-a29c-81e4b4f238a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:12.067692 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e195378f-6c59-4c09-a29c-81e4b4f238a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:12.067977 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e195378f-6c59-4c09-a29c-81e4b4f238a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:12.068258 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e195378f-6c59-4c09-a29c-81e4b4f238a4 admin admin] PATH_INFO: `/v3/projects/5ba2499f30044560b6eeb26af6019a51/users/57a98c89932145f38c22a86bf9d34766/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:12.070280 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e195378f-6c59-4c09-a29c-81e4b4f238a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ba2499f30044560b6eeb26af6019a51, user_id=57a98c89932145f38c22a86bf9d34766, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:12.072935 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e195378f-6c59-4c09-a29c-81e4b4f238a4 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:12.083178 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1344/2706] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:15:12 2022] PUT /identity/v3/projects/5ba2499f30044560b6eeb26af6019a51/users/57a98c89932145f38c22a86bf9d34766/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:15:12.093210 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc7588-4dd4-4d40-9b9c-366686ea59f0 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:12.093498 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc7588-4dd4-4d40-9b9c-366686ea59f0 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:12.093822 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc7588-4dd4-4d40-9b9c-366686ea59f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:12.120415 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-5cdc7588-4dd4-4d40-9b9c-366686ea59f0 None None] MFA Rules not processed for user `57a98c89932145f38c22a86bf9d34766`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:12.121619 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-5cdc7588-4dd4-4d40-9b9c-366686ea59f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:12.187715 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1363/2707] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:15:12 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:15:12.199814 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e195378f-6c59-4c09-a29c-81e4b4f238a4 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:12.240876 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:12.256410 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e195378f-6c59-4c09-a29c-81e4b4f238a4 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:12.260178 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1466e408-6237-40ed-9d52-b1f21e5fe13d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:12.260860 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1466e408-6237-40ed-9d52-b1f21e5fe13d service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:12.261166 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1466e408-6237-40ed-9d52-b1f21e5fe13d service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:12.261443 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1466e408-6237-40ed-9d52-b1f21e5fe13d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:12.270521 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1466e408-6237-40ed-9d52-b1f21e5fe13d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:12.272002 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1466e408-6237-40ed-9d52-b1f21e5fe13d service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:12.281260 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1345/2708] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:15:12 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:15:12.321547 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc7588-4dd4-4d40-9b9c-366686ea59f0 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:12.322881 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-5cdc7588-4dd4-4d40-9b9c-366686ea59f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:12.334622 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc7588-4dd4-4d40-9b9c-366686ea59f0 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:12.338165 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac314a4-9b7d-4357-adb4-410452523b01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:12.338798 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac314a4-9b7d-4357-adb4-410452523b01 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:12.339084 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac314a4-9b7d-4357-adb4-410452523b01 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:12.339396 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac314a4-9b7d-4357-adb4-410452523b01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:12.395287 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac314a4-9b7d-4357-adb4-410452523b01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:12.396737 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac314a4-9b7d-4357-adb4-410452523b01 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:12.405374 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1364/2709] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:15:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:15:14.239747 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1466e408-6237-40ed-9d52-b1f21e5fe13d service cinder] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:14.301708 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1466e408-6237-40ed-9d52-b1f21e5fe13d service cinder] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:14.306185 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f02229d-81f7-442e-a837-c13813ad3754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:14.307065 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02229d-81f7-442e-a837-c13813ad3754 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:14.307370 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02229d-81f7-442e-a837-c13813ad3754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:14.307649 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02229d-81f7-442e-a837-c13813ad3754 admin admin] PATH_INFO: `/v3/users/ed8e3ca088c4452689969585b8ce6fd8` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:14.308832 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f02229d-81f7-442e-a837-c13813ad3754 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed8e3ca088c4452689969585b8ce6fd8)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:14.310741 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f02229d-81f7-442e-a837-c13813ad3754 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:14.357040 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-8f02229d-81f7-442e-a837-c13813ad3754 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed8e3ca088c4452689969585b8ce6fd8'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:14.357677 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-8f02229d-81f7-442e-a837-c13813ad3754 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed8e3ca088c4452689969585b8ce6fd8'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:14.365971 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-8f02229d-81f7-442e-a837-c13813ad3754 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed8e3ca088c4452689969585b8ce6fd8'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:14.378147 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-8f02229d-81f7-442e-a837-c13813ad3754 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed8e3ca088c4452689969585b8ce6fd8'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:14.386121 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-8f02229d-81f7-442e-a837-c13813ad3754 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed8e3ca088c4452689969585b8ce6fd8'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:14.397033 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1346/2710] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:15:14 2022] DELETE /identity/v3/users/ed8e3ca088c4452689969585b8ce6fd8 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:15:14.400826 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf640e6b-3de9-4ca7-95dd-dc13952d02d8 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:14.401707 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf640e6b-3de9-4ca7-95dd-dc13952d02d8 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:14.401707 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf640e6b-3de9-4ca7-95dd-dc13952d02d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:14.420001 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-cf640e6b-3de9-4ca7-95dd-dc13952d02d8 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:14.421590 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-cf640e6b-3de9-4ca7-95dd-dc13952d02d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:14.493425 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1365/2711] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:15:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:15:14.500059 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f02229d-81f7-442e-a837-c13813ad3754 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:14.501591 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-8f02229d-81f7-442e-a837-c13813ad3754 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:14.560542 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f02229d-81f7-442e-a837-c13813ad3754 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:14.564148 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663daaf6-8141-4cb0-a7e8-4d85eac3348e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:14.564788 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663daaf6-8141-4cb0-a7e8-4d85eac3348e service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:14.565072 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663daaf6-8141-4cb0-a7e8-4d85eac3348e service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:14.565954 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663daaf6-8141-4cb0-a7e8-4d85eac3348e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:14.574795 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663daaf6-8141-4cb0-a7e8-4d85eac3348e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:14.576234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663daaf6-8141-4cb0-a7e8-4d85eac3348e service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:14.586287 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1347/2712] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:15:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:15:14.865073 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf640e6b-3de9-4ca7-95dd-dc13952d02d8 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:14.866571 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-cf640e6b-3de9-4ca7-95dd-dc13952d02d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:14.876165 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:14.881212 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf640e6b-3de9-4ca7-95dd-dc13952d02d8 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:14.886894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52502c3f-60de-4615-a9ed-591e7b8ce776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:14.887922 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52502c3f-60de-4615-a9ed-591e7b8ce776 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:14.888120 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52502c3f-60de-4615-a9ed-591e7b8ce776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:14.888401 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52502c3f-60de-4615-a9ed-591e7b8ce776 admin admin] PATH_INFO: `/v3/projects/b21e9ae591604b779cbb7ad17f6b3fb5` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:14.889652 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52502c3f-60de-4615-a9ed-591e7b8ce776 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b21e9ae591604b779cbb7ad17f6b3fb5)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:14.891599 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52502c3f-60de-4615-a9ed-591e7b8ce776 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:14.937870 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-52502c3f-60de-4615-a9ed-591e7b8ce776 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b21e9ae591604b779cbb7ad17f6b3fb5', 'request_id': 'req-52502c3f-60de-4615-a9ed-591e7b8ce776'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:14.946223 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-52502c3f-60de-4615-a9ed-591e7b8ce776 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b21e9ae591604b779cbb7ad17f6b3fb5', 'request_id': 'req-52502c3f-60de-4615-a9ed-591e7b8ce776'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:14.951754 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-52502c3f-60de-4615-a9ed-591e7b8ce776 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b21e9ae591604b779cbb7ad17f6b3fb5', 'request_id': 'req-52502c3f-60de-4615-a9ed-591e7b8ce776'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:14.962466 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-52502c3f-60de-4615-a9ed-591e7b8ce776 admin admin] The token cache is being invalidate because project b21e9ae591604b779cbb7ad17f6b3fb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:15:14.962750 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-52502c3f-60de-4615-a9ed-591e7b8ce776 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:14.963402 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-52502c3f-60de-4615-a9ed-591e7b8ce776 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:14.964910 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1366/2713] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:15:14 2022] DELETE /identity/v3/projects/b21e9ae591604b779cbb7ad17f6b3fb5 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:15:14.968903 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663daaf6-8141-4cb0-a7e8-4d85eac3348e service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:14.973765 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:14.980708 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663daaf6-8141-4cb0-a7e8-4d85eac3348e service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:15.037687 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679bb78e-47e8-4933-98f7-4bea642dacee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:15.038549 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679bb78e-47e8-4933-98f7-4bea642dacee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:15.038829 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679bb78e-47e8-4933-98f7-4bea642dacee admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:15.039106 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679bb78e-47e8-4933-98f7-4bea642dacee admin admin] PATH_INFO: `/v3/users/cf1c97847ab74097a33597708ab86aac` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:15.040327 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679bb78e-47e8-4933-98f7-4bea642dacee admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf1c97847ab74097a33597708ab86aac)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:15.042231 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679bb78e-47e8-4933-98f7-4bea642dacee admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:15.096726 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-679bb78e-47e8-4933-98f7-4bea642dacee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf1c97847ab74097a33597708ab86aac'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:15.097458 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-679bb78e-47e8-4933-98f7-4bea642dacee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf1c97847ab74097a33597708ab86aac'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:15.106229 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-679bb78e-47e8-4933-98f7-4bea642dacee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf1c97847ab74097a33597708ab86aac'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:15.121938 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-679bb78e-47e8-4933-98f7-4bea642dacee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf1c97847ab74097a33597708ab86aac'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:15.129497 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-679bb78e-47e8-4933-98f7-4bea642dacee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf1c97847ab74097a33597708ab86aac'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:15.135387 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1348/2714] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:15:14 2022] DELETE /identity/v3/users/cf1c97847ab74097a33597708ab86aac => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:15:15.243240 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52502c3f-60de-4615-a9ed-591e7b8ce776 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:15.244692 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-52502c3f-60de-4615-a9ed-591e7b8ce776 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:15.258199 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52502c3f-60de-4615-a9ed-591e7b8ce776 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:15.318487 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a378b13-c1e3-4cc5-b23d-7d39901ced6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:15.319698 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a378b13-c1e3-4cc5-b23d-7d39901ced6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:15.319982 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a378b13-c1e3-4cc5-b23d-7d39901ced6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:15.320260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a378b13-c1e3-4cc5-b23d-7d39901ced6b admin admin] PATH_INFO: `/v3/users/c32abf74f4de4519a8b7552f70ed4192` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:15.321467 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a378b13-c1e3-4cc5-b23d-7d39901ced6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c32abf74f4de4519a8b7552f70ed4192)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:15.323403 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a378b13-c1e3-4cc5-b23d-7d39901ced6b admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:15.370901 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-0a378b13-c1e3-4cc5-b23d-7d39901ced6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c32abf74f4de4519a8b7552f70ed4192'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:15.378419 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-0a378b13-c1e3-4cc5-b23d-7d39901ced6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c32abf74f4de4519a8b7552f70ed4192'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:15.391110 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-0a378b13-c1e3-4cc5-b23d-7d39901ced6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c32abf74f4de4519a8b7552f70ed4192'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:15.391905 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-0a378b13-c1e3-4cc5-b23d-7d39901ced6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c32abf74f4de4519a8b7552f70ed4192'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:15.392524 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-0a378b13-c1e3-4cc5-b23d-7d39901ced6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c32abf74f4de4519a8b7552f70ed4192'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:15.411675 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1367/2715] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:15:15 2022] DELETE /identity/v3/users/c32abf74f4de4519a8b7552f70ed4192 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:15:15.416415 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b07496e-015c-49a9-a290-77ede8b35e7e None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:15.417204 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b07496e-015c-49a9-a290-77ede8b35e7e None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:15.417204 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b07496e-015c-49a9-a290-77ede8b35e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:15.432060 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a378b13-c1e3-4cc5-b23d-7d39901ced6b admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:15.433574 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-0a378b13-c1e3-4cc5-b23d-7d39901ced6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:15.437199 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-0b07496e-015c-49a9-a290-77ede8b35e7e None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:15.438382 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-0b07496e-015c-49a9-a290-77ede8b35e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:15.444472 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:15.506083 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1349/2716] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:15:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:15:15.507420 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a378b13-c1e3-4cc5-b23d-7d39901ced6b admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:15.512336 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42354df4-5339-463f-a1be-624a5e327d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:15.513289 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42354df4-5339-463f-a1be-624a5e327d00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:15.513577 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42354df4-5339-463f-a1be-624a5e327d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:15.513854 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42354df4-5339-463f-a1be-624a5e327d00 admin admin] PATH_INFO: `/v3/projects/e721ac34a87a41c38328e398e34eb16a` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:15.515078 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42354df4-5339-463f-a1be-624a5e327d00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e721ac34a87a41c38328e398e34eb16a)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:15.515610 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b07496e-015c-49a9-a290-77ede8b35e7e None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:15.516938 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-0b07496e-015c-49a9-a290-77ede8b35e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:15.517073 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42354df4-5339-463f-a1be-624a5e327d00 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:15.544073 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:15.567638 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-42354df4-5339-463f-a1be-624a5e327d00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e721ac34a87a41c38328e398e34eb16a', 'request_id': 'req-42354df4-5339-463f-a1be-624a5e327d00'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:15.575157 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-42354df4-5339-463f-a1be-624a5e327d00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e721ac34a87a41c38328e398e34eb16a', 'request_id': 'req-42354df4-5339-463f-a1be-624a5e327d00'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:15.580221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b07496e-015c-49a9-a290-77ede8b35e7e None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:15.581981 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-42354df4-5339-463f-a1be-624a5e327d00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e721ac34a87a41c38328e398e34eb16a', 'request_id': 'req-42354df4-5339-463f-a1be-624a5e327d00'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:15.594111 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-42354df4-5339-463f-a1be-624a5e327d00 admin admin] The token cache is being invalidate because project e721ac34a87a41c38328e398e34eb16a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:15:15.594393 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-42354df4-5339-463f-a1be-624a5e327d00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:15.595069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-42354df4-5339-463f-a1be-624a5e327d00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:15.602157 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1368/2717] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:15:15 2022] DELETE /identity/v3/projects/e721ac34a87a41c38328e398e34eb16a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:15:15.603939 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c1ceab-8709-438f-bd57-7b5275aac508 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:15.604225 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c1ceab-8709-438f-bd57-7b5275aac508 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:15.604502 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c1ceab-8709-438f-bd57-7b5275aac508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:15.624761 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-d4c1ceab-8709-438f-bd57-7b5275aac508 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:15.625956 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d4c1ceab-8709-438f-bd57-7b5275aac508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:15.631719 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bedc84a-f995-4bb2-9279-2a6d71b34303 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:15.632377 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bedc84a-f995-4bb2-9279-2a6d71b34303 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:15.632661 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bedc84a-f995-4bb2-9279-2a6d71b34303 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:15.632945 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bedc84a-f995-4bb2-9279-2a6d71b34303 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:15.634800 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-9bedc84a-f995-4bb2-9279-2a6d71b34303 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:15.684823 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1369/2718] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:15:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:15:15.692608 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e21d5a1-dd48-4f9e-805c-e5312756546c None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:15.692929 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e21d5a1-dd48-4f9e-805c-e5312756546c None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:15.693215 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e21d5a1-dd48-4f9e-805c-e5312756546c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:15.702379 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bedc84a-f995-4bb2-9279-2a6d71b34303 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:15.704148 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bedc84a-f995-4bb2-9279-2a6d71b34303 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:15.711159 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-0e21d5a1-dd48-4f9e-805c-e5312756546c None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:15.712403 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-0e21d5a1-dd48-4f9e-805c-e5312756546c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:15.716953 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1350/2719] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:15:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 22:15:15.720894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1370/2720] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:15:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:15:15.725962 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bedc84a-f995-4bb2-9279-2a6d71b34303 service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:15.737774 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bedc84a-f995-4bb2-9279-2a6d71b34303 service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:15.743592 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcfc95f-4dbd-4193-ab65-a7fea59cb402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:15.744704 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcfc95f-4dbd-4193-ab65-a7fea59cb402 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:15.745074 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcfc95f-4dbd-4193-ab65-a7fea59cb402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:15.745364 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcfc95f-4dbd-4193-ab65-a7fea59cb402 admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:15.746075 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcfc95f-4dbd-4193-ab65-a7fea59cb402 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:15:15.746376 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcfc95f-4dbd-4193-ab65-a7fea59cb402 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:15.748020 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcfc95f-4dbd-4193-ab65-a7fea59cb402 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:15.761695 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1351/2721] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:15:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:15:15.768841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e21d5a1-dd48-4f9e-805c-e5312756546c None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:15.780469 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e21d5a1-dd48-4f9e-805c-e5312756546c None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:15.786173 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbffa106-252e-4fe4-bbf0-7506d8f22da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:15.786934 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbffa106-252e-4fe4-bbf0-7506d8f22da2 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:15.787351 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbffa106-252e-4fe4-bbf0-7506d8f22da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:15.787767 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbffa106-252e-4fe4-bbf0-7506d8f22da2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:15.788554 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbffa106-252e-4fe4-bbf0-7506d8f22da2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:15.790598 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbffa106-252e-4fe4-bbf0-7506d8f22da2 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:15.818037 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1371/2722] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:15:15 2022] POST /identity/v3/projects => generated 391 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:15:15.822355 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcfc95f-4dbd-4193-ab65-a7fea59cb402 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:15.833637 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcfc95f-4dbd-4193-ab65-a7fea59cb402 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:15.903746 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d6f775-3925-490c-8bd9-eade567de8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:15.904418 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d6f775-3925-490c-8bd9-eade567de8f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:15.904701 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d6f775-3925-490c-8bd9-eade567de8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:15.904985 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d6f775-3925-490c-8bd9-eade567de8f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:15.905682 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d6f775-3925-490c-8bd9-eade567de8f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:15.907606 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d6f775-3925-490c-8bd9-eade567de8f9 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:15.929142 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1352/2723] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:15:15 2022] POST /identity/v3/users => generated 296 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:15:15.933090 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbffa106-252e-4fe4-bbf0-7506d8f22da2 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:15.943655 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbffa106-252e-4fe4-bbf0-7506d8f22da2 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:15.948289 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c23eb8-6cc1-405d-aada-09df2f0389f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:15.948947 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c23eb8-6cc1-405d-aada-09df2f0389f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:15.949228 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c23eb8-6cc1-405d-aada-09df2f0389f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:15.949505 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c23eb8-6cc1-405d-aada-09df2f0389f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:15.950222 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c23eb8-6cc1-405d-aada-09df2f0389f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:15.951847 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c23eb8-6cc1-405d-aada-09df2f0389f3 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:15.954384 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:15.967745 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1372/2724] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:15:15.968603 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d6f775-3925-490c-8bd9-eade567de8f9 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:15.975470 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:15.980849 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d6f775-3925-490c-8bd9-eade567de8f9 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:15.988699 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b5551f-f5f6-4434-a70a-ce97256e46bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:15.989357 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5551f-f5f6-4434-a70a-ce97256e46bc admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:15.989645 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5551f-f5f6-4434-a70a-ce97256e46bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:15.989929 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5551f-f5f6-4434-a70a-ce97256e46bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:15.990663 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b5551f-f5f6-4434-a70a-ce97256e46bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:15.992542 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b5551f-f5f6-4434-a70a-ce97256e46bc admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:16.003144 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c23eb8-6cc1-405d-aada-09df2f0389f3 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:16.004273 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1353/2725] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 22:15:16.008215 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b5551f-f5f6-4434-a70a-ce97256e46bc admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:16.008630 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-66c23eb8-6cc1-405d-aada-09df2f0389f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:16.020234 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b5551f-f5f6-4434-a70a-ce97256e46bc admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:16.022581 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c23eb8-6cc1-405d-aada-09df2f0389f3 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:16.025846 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51af657-c02b-4b03-9cff-3334d9423299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:16.027608 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51af657-c02b-4b03-9cff-3334d9423299 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:16.027895 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51af657-c02b-4b03-9cff-3334d9423299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:16.028174 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51af657-c02b-4b03-9cff-3334d9423299 admin admin] PATH_INFO: `/v3/projects/a5f0a610b30645dc98ca47b9581e32b8/users/96c957909dc6483f9c8946e70807d791/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:16.028293 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb4b740-774e-4fce-8c68-c52ca47096c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:16.029292 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4b740-774e-4fce-8c68-c52ca47096c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:16.029726 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4b740-774e-4fce-8c68-c52ca47096c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:16.030026 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4b740-774e-4fce-8c68-c52ca47096c8 admin admin] PATH_INFO: `/v3/projects/cdb7b4f88370455baff2e9ee434700fe` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:16.031370 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb4b740-774e-4fce-8c68-c52ca47096c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdb7b4f88370455baff2e9ee434700fe)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:16.033525 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb4b740-774e-4fce-8c68-c52ca47096c8 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:16.048620 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51af657-c02b-4b03-9cff-3334d9423299 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5f0a610b30645dc98ca47b9581e32b8, user_id=96c957909dc6483f9c8946e70807d791, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:16.056790 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51af657-c02b-4b03-9cff-3334d9423299 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:16.073518 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1354/2726] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:15:16 2022] PUT /identity/v3/projects/a5f0a610b30645dc98ca47b9581e32b8/users/96c957909dc6483f9c8946e70807d791/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:15:16.086610 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-6bb4b740-774e-4fce-8c68-c52ca47096c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdb7b4f88370455baff2e9ee434700fe', 'request_id': 'req-6bb4b740-774e-4fce-8c68-c52ca47096c8'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:16.086848 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa08231-709f-4974-8244-80c8aec40ac3 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:16.087089 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa08231-709f-4974-8244-80c8aec40ac3 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:16.087263 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa08231-709f-4974-8244-80c8aec40ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:16.095552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-6bb4b740-774e-4fce-8c68-c52ca47096c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdb7b4f88370455baff2e9ee434700fe', 'request_id': 'req-6bb4b740-774e-4fce-8c68-c52ca47096c8'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:16.101255 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-6bb4b740-774e-4fce-8c68-c52ca47096c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdb7b4f88370455baff2e9ee434700fe', 'request_id': 'req-6bb4b740-774e-4fce-8c68-c52ca47096c8'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:16.112523 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-6bb4b740-774e-4fce-8c68-c52ca47096c8 admin admin] The token cache is being invalidate because project cdb7b4f88370455baff2e9ee434700fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:15:16.112810 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-6bb4b740-774e-4fce-8c68-c52ca47096c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:16.113480 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-6bb4b740-774e-4fce-8c68-c52ca47096c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:16.115026 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1373/2727] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:15:16 2022] DELETE /identity/v3/projects/cdb7b4f88370455baff2e9ee434700fe => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:15:16.118172 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-5fa08231-709f-4974-8244-80c8aec40ac3 None None] MFA Rules not processed for user `96c957909dc6483f9c8946e70807d791`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:16.118172 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-5fa08231-709f-4974-8244-80c8aec40ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:16.119050 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb4b740-774e-4fce-8c68-c52ca47096c8 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:16.120350 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-6bb4b740-774e-4fce-8c68-c52ca47096c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:16.182790 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1355/2728] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:15:16 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:15:16.185500 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb4b740-774e-4fce-8c68-c52ca47096c8 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:16.191306 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61e6cd4-9662-4ad1-9aeb-f066c6792b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:16.192220 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61e6cd4-9662-4ad1-9aeb-f066c6792b39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:16.192505 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61e6cd4-9662-4ad1-9aeb-f066c6792b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:16.192857 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61e6cd4-9662-4ad1-9aeb-f066c6792b39 admin admin] PATH_INFO: `/v3/users/194bd33031e2401f9d5862bcd38257bb` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:16.193989 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa08231-709f-4974-8244-80c8aec40ac3 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:16.194123 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61e6cd4-9662-4ad1-9aeb-f066c6792b39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=194bd33031e2401f9d5862bcd38257bb)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:16.195425 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-5fa08231-709f-4974-8244-80c8aec40ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:16.196004 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61e6cd4-9662-4ad1-9aeb-f066c6792b39 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:16.246583 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-f61e6cd4-9662-4ad1-9aeb-f066c6792b39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '194bd33031e2401f9d5862bcd38257bb'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:16.254162 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-f61e6cd4-9662-4ad1-9aeb-f066c6792b39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '194bd33031e2401f9d5862bcd38257bb'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:16.255468 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa08231-709f-4974-8244-80c8aec40ac3 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:16.259172 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5408c1b-f37f-4e53-897a-f7e348932ac9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:16.259871 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5408c1b-f37f-4e53-897a-f7e348932ac9 service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:16.260156 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5408c1b-f37f-4e53-897a-f7e348932ac9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:16.260427 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5408c1b-f37f-4e53-897a-f7e348932ac9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:16.267557 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-f61e6cd4-9662-4ad1-9aeb-f066c6792b39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '194bd33031e2401f9d5862bcd38257bb'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:16.268242 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-f61e6cd4-9662-4ad1-9aeb-f066c6792b39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '194bd33031e2401f9d5862bcd38257bb'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:16.269143 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5408c1b-f37f-4e53-897a-f7e348932ac9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:16.269757 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-f61e6cd4-9662-4ad1-9aeb-f066c6792b39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '194bd33031e2401f9d5862bcd38257bb'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:16.270547 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5408c1b-f37f-4e53-897a-f7e348932ac9 service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:16.279614 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:16.279980 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1356/2729] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:15:16 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:15:16.288334 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1374/2730] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:15:16 2022] DELETE /identity/v3/users/194bd33031e2401f9d5862bcd38257bb => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:15:16.598194 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5408c1b-f37f-4e53-897a-f7e348932ac9 service cinder] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:16.619704 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5408c1b-f37f-4e53-897a-f7e348932ac9 service cinder] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:16.679855 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35fba7e-4d3a-4771-9cd2-4d3ccbf19153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:16.680712 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35fba7e-4d3a-4771-9cd2-4d3ccbf19153 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:16.680987 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35fba7e-4d3a-4771-9cd2-4d3ccbf19153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:16.681258 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35fba7e-4d3a-4771-9cd2-4d3ccbf19153 admin admin] PATH_INFO: `/v3/projects/8580534b941b4617b38d4d3d41f353e4` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:16.682423 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35fba7e-4d3a-4771-9cd2-4d3ccbf19153 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8580534b941b4617b38d4d3d41f353e4)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:16.684461 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35fba7e-4d3a-4771-9cd2-4d3ccbf19153 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:16.686793 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:16.732967 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-a35fba7e-4d3a-4771-9cd2-4d3ccbf19153 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8580534b941b4617b38d4d3d41f353e4', 'request_id': 'req-a35fba7e-4d3a-4771-9cd2-4d3ccbf19153'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:16.740523 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-a35fba7e-4d3a-4771-9cd2-4d3ccbf19153 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8580534b941b4617b38d4d3d41f353e4', 'request_id': 'req-a35fba7e-4d3a-4771-9cd2-4d3ccbf19153'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:16.746774 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-a35fba7e-4d3a-4771-9cd2-4d3ccbf19153 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8580534b941b4617b38d4d3d41f353e4', 'request_id': 'req-a35fba7e-4d3a-4771-9cd2-4d3ccbf19153'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:16.757986 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-a35fba7e-4d3a-4771-9cd2-4d3ccbf19153 admin admin] The token cache is being invalidate because project 8580534b941b4617b38d4d3d41f353e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:15:16.758283 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-a35fba7e-4d3a-4771-9cd2-4d3ccbf19153 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:16.758954 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-a35fba7e-4d3a-4771-9cd2-4d3ccbf19153 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:16.760537 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1357/2731] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:15:16 2022] DELETE /identity/v3/projects/8580534b941b4617b38d4d3d41f353e4 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:15:16.764574 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61e6cd4-9662-4ad1-9aeb-f066c6792b39 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:16.765968 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-f61e6cd4-9662-4ad1-9aeb-f066c6792b39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:16.779094 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61e6cd4-9662-4ad1-9aeb-f066c6792b39 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:16.834363 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009e13bb-d4bd-47a2-8750-212bbcbd5f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:16.835250 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e13bb-d4bd-47a2-8750-212bbcbd5f27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:16.835550 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e13bb-d4bd-47a2-8750-212bbcbd5f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:16.835829 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e13bb-d4bd-47a2-8750-212bbcbd5f27 admin admin] PATH_INFO: `/v3/users/57a98c89932145f38c22a86bf9d34766` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:16.837017 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009e13bb-d4bd-47a2-8750-212bbcbd5f27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57a98c89932145f38c22a86bf9d34766)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:16.838924 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009e13bb-d4bd-47a2-8750-212bbcbd5f27 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:16.884287 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-009e13bb-d4bd-47a2-8750-212bbcbd5f27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57a98c89932145f38c22a86bf9d34766'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:16.890999 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-009e13bb-d4bd-47a2-8750-212bbcbd5f27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57a98c89932145f38c22a86bf9d34766'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:16.902716 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-009e13bb-d4bd-47a2-8750-212bbcbd5f27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57a98c89932145f38c22a86bf9d34766'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:16.903406 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-009e13bb-d4bd-47a2-8750-212bbcbd5f27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57a98c89932145f38c22a86bf9d34766'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:16.904013 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-009e13bb-d4bd-47a2-8750-212bbcbd5f27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57a98c89932145f38c22a86bf9d34766'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:16.917872 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1375/2732] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:15:16 2022] DELETE /identity/v3/users/57a98c89932145f38c22a86bf9d34766 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:15:17.183200 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35fba7e-4d3a-4771-9cd2-4d3ccbf19153 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:17.184747 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-a35fba7e-4d3a-4771-9cd2-4d3ccbf19153 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:17.198724 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35fba7e-4d3a-4771-9cd2-4d3ccbf19153 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:17.256534 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9150bb26-8bac-4c91-84dc-b600a277dd9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:17.257392 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150bb26-8bac-4c91-84dc-b600a277dd9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:17.257676 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150bb26-8bac-4c91-84dc-b600a277dd9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:17.257983 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150bb26-8bac-4c91-84dc-b600a277dd9c admin admin] PATH_INFO: `/v3/projects/5ba2499f30044560b6eeb26af6019a51` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:17.259169 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9150bb26-8bac-4c91-84dc-b600a277dd9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ba2499f30044560b6eeb26af6019a51)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:17.261118 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9150bb26-8bac-4c91-84dc-b600a277dd9c admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:17.307790 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-9150bb26-8bac-4c91-84dc-b600a277dd9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ba2499f30044560b6eeb26af6019a51', 'request_id': 'req-9150bb26-8bac-4c91-84dc-b600a277dd9c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:17.314900 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-9150bb26-8bac-4c91-84dc-b600a277dd9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ba2499f30044560b6eeb26af6019a51', 'request_id': 'req-9150bb26-8bac-4c91-84dc-b600a277dd9c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:17.321094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-9150bb26-8bac-4c91-84dc-b600a277dd9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ba2499f30044560b6eeb26af6019a51', 'request_id': 'req-9150bb26-8bac-4c91-84dc-b600a277dd9c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:17.331932 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-9150bb26-8bac-4c91-84dc-b600a277dd9c admin admin] The token cache is being invalidate because project 5ba2499f30044560b6eeb26af6019a51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:15:17.332123 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-9150bb26-8bac-4c91-84dc-b600a277dd9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:17.332736 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-9150bb26-8bac-4c91-84dc-b600a277dd9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:17.334248 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1358/2733] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:15:17 2022] DELETE /identity/v3/projects/5ba2499f30044560b6eeb26af6019a51 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:15:17.340141 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b5722-215c-402b-b332-371a98b7f917 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:17.340458 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b5722-215c-402b-b332-371a98b7f917 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:17.340734 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b5722-215c-402b-b332-371a98b7f917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:17.359687 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-475b5722-215c-402b-b332-371a98b7f917 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:17.360872 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-475b5722-215c-402b-b332-371a98b7f917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:17.421458 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1376/2734] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:15:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:15:17.429971 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8811a-4139-41fe-8077-bd21a61a245d None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:17.430264 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8811a-4139-41fe-8077-bd21a61a245d None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:17.430547 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8811a-4139-41fe-8077-bd21a61a245d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:17.448529 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-fbe8811a-4139-41fe-8077-bd21a61a245d None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:17.449734 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-fbe8811a-4139-41fe-8077-bd21a61a245d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:17.458063 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1359/2735] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:15:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:15:17.462280 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b5722-215c-402b-b332-371a98b7f917 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:17.473052 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b5722-215c-402b-b332-371a98b7f917 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:17.477564 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bedcae-c101-4843-829b-7f2e3acac02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:17.478489 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bedcae-c101-4843-829b-7f2e3acac02b admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:17.478880 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bedcae-c101-4843-829b-7f2e3acac02b admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:17.479152 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bedcae-c101-4843-829b-7f2e3acac02b admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:17.479900 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bedcae-c101-4843-829b-7f2e3acac02b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:15:17.480202 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bedcae-c101-4843-829b-7f2e3acac02b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:17.481775 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bedcae-c101-4843-829b-7f2e3acac02b admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:17.494187 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1377/2736] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:15:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 22:15:17.498414 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe8811a-4139-41fe-8077-bd21a61a245d None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:17.507605 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe8811a-4139-41fe-8077-bd21a61a245d None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:17.512205 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d54131f-5ec9-484d-b8a0-ca8a138bf465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:17.512873 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d54131f-5ec9-484d-b8a0-ca8a138bf465 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:17.513158 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d54131f-5ec9-484d-b8a0-ca8a138bf465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:17.513435 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d54131f-5ec9-484d-b8a0-ca8a138bf465 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:17.514128 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d54131f-5ec9-484d-b8a0-ca8a138bf465 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:17.516041 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d54131f-5ec9-484d-b8a0-ca8a138bf465 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:17.529211 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:17.543045 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1360/2737] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:15:17 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:15:17.547238 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bedcae-c101-4843-829b-7f2e3acac02b admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:17.559173 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bedcae-c101-4843-829b-7f2e3acac02b admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:17.562810 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:17.621258 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d621d253-8ca1-4f0e-8883-bf5e7083f8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:17.621928 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621d253-8ca1-4f0e-8883-bf5e7083f8c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:17.622209 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621d253-8ca1-4f0e-8883-bf5e7083f8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:17.622503 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621d253-8ca1-4f0e-8883-bf5e7083f8c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:17.623184 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d621d253-8ca1-4f0e-8883-bf5e7083f8c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:17.625097 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d621d253-8ca1-4f0e-8883-bf5e7083f8c0 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:17.648209 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1378/2738] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:15:17 2022] POST /identity/v3/users => generated 286 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:15:17.653731 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d54131f-5ec9-484d-b8a0-ca8a138bf465 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:17.664368 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d54131f-5ec9-484d-b8a0-ca8a138bf465 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:17.668930 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f77cc84-7f11-450a-a6e3-40d3a3af7db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:17.669595 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77cc84-7f11-450a-a6e3-40d3a3af7db2 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:17.669881 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77cc84-7f11-450a-a6e3-40d3a3af7db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:17.670160 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77cc84-7f11-450a-a6e3-40d3a3af7db2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:17.670883 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f77cc84-7f11-450a-a6e3-40d3a3af7db2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:17.672562 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f77cc84-7f11-450a-a6e3-40d3a3af7db2 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:17.683274 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1361/2739] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 22:15:17.687698 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d621d253-8ca1-4f0e-8883-bf5e7083f8c0 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:17.700119 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d621d253-8ca1-4f0e-8883-bf5e7083f8c0 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:17.706360 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbac349f-3d92-4877-b9de-bb517afd075f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:17.707386 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac349f-3d92-4877-b9de-bb517afd075f admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:17.707670 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac349f-3d92-4877-b9de-bb517afd075f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:17.707942 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac349f-3d92-4877-b9de-bb517afd075f admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:17.708667 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbac349f-3d92-4877-b9de-bb517afd075f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:17.710275 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbac349f-3d92-4877-b9de-bb517afd075f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:17.720778 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1379/2740] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:15:17.725043 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f77cc84-7f11-450a-a6e3-40d3a3af7db2 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:17.735786 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f77cc84-7f11-450a-a6e3-40d3a3af7db2 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:17.741001 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3372e85f-9c8d-4844-a793-25a1c57bd214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:17.741890 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3372e85f-9c8d-4844-a793-25a1c57bd214 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:17.742174 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3372e85f-9c8d-4844-a793-25a1c57bd214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:17.742456 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3372e85f-9c8d-4844-a793-25a1c57bd214 admin admin] PATH_INFO: `/v3/projects/81781a7c2c1a4de694fc4fd498475ba4/users/f22c0325dd044014ab911fe2a0ca2973/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:17.758816 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3372e85f-9c8d-4844-a793-25a1c57bd214 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81781a7c2c1a4de694fc4fd498475ba4, user_id=f22c0325dd044014ab911fe2a0ca2973, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:17.761542 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3372e85f-9c8d-4844-a793-25a1c57bd214 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:17.777104 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1362/2741] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:15:17 2022] PUT /identity/v3/projects/81781a7c2c1a4de694fc4fd498475ba4/users/f22c0325dd044014ab911fe2a0ca2973/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:15:17.788815 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a6d2a-51ea-4e47-b392-9618840eee3e None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:17.789311 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a6d2a-51ea-4e47-b392-9618840eee3e None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:17.789518 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a6d2a-51ea-4e47-b392-9618840eee3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:17.821709 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-850a6d2a-51ea-4e47-b392-9618840eee3e None None] MFA Rules not processed for user `f22c0325dd044014ab911fe2a0ca2973`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:17.822813 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-850a6d2a-51ea-4e47-b392-9618840eee3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:17.888576 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1380/2742] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:15:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:15:17.900692 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3372e85f-9c8d-4844-a793-25a1c57bd214 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:17.902406 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-3372e85f-9c8d-4844-a793-25a1c57bd214 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:17.959414 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3372e85f-9c8d-4844-a793-25a1c57bd214 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:17.963022 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a350a860-3eed-48be-969d-cbcce950e780 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:17.963713 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350a860-3eed-48be-969d-cbcce950e780 service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:17.964001 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350a860-3eed-48be-969d-cbcce950e780 service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:17.964281 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350a860-3eed-48be-969d-cbcce950e780 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:17.972694 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a350a860-3eed-48be-969d-cbcce950e780 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:17.974110 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a350a860-3eed-48be-969d-cbcce950e780 service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:17.983010 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1363/2743] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:15:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:15:18.002394 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a6d2a-51ea-4e47-b392-9618840eee3e None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:18.063114 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a6d2a-51ea-4e47-b392-9618840eee3e None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:18.068358 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579e27cc-b277-4621-b671-d6c05cb1ae4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:18.069224 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e27cc-b277-4621-b671-d6c05cb1ae4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:18.069513 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e27cc-b277-4621-b671-d6c05cb1ae4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:18.069789 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e27cc-b277-4621-b671-d6c05cb1ae4f admin admin] PATH_INFO: `/v3/users/f22c0325dd044014ab911fe2a0ca2973` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:18.070985 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579e27cc-b277-4621-b671-d6c05cb1ae4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f22c0325dd044014ab911fe2a0ca2973)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:18.072930 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579e27cc-b277-4621-b671-d6c05cb1ae4f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:18.087853 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:18.119055 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-579e27cc-b277-4621-b671-d6c05cb1ae4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f22c0325dd044014ab911fe2a0ca2973'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:18.128642 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-579e27cc-b277-4621-b671-d6c05cb1ae4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f22c0325dd044014ab911fe2a0ca2973'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:18.143332 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-579e27cc-b277-4621-b671-d6c05cb1ae4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f22c0325dd044014ab911fe2a0ca2973'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:18.144126 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-579e27cc-b277-4621-b671-d6c05cb1ae4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f22c0325dd044014ab911fe2a0ca2973'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:18.144754 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-579e27cc-b277-4621-b671-d6c05cb1ae4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f22c0325dd044014ab911fe2a0ca2973'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:18.159274 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1381/2744] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:15:18 2022] DELETE /identity/v3/users/f22c0325dd044014ab911fe2a0ca2973 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:15:18.164902 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae925a5-184f-4914-aec2-b0ffc2c257d0 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:18.165191 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae925a5-184f-4914-aec2-b0ffc2c257d0 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:18.165468 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae925a5-184f-4914-aec2-b0ffc2c257d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:18.183471 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-5ae925a5-184f-4914-aec2-b0ffc2c257d0 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:18.184676 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-5ae925a5-184f-4914-aec2-b0ffc2c257d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:18.243299 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1364/2745] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:15:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:15:18.253444 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579e27cc-b277-4621-b671-d6c05cb1ae4f admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:18.254862 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-579e27cc-b277-4621-b671-d6c05cb1ae4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:18.312055 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579e27cc-b277-4621-b671-d6c05cb1ae4f admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:18.316321 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93412c52-f7f2-4d1a-8fbb-da3b27445aa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:18.316985 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93412c52-f7f2-4d1a-8fbb-da3b27445aa6 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:18.317272 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93412c52-f7f2-4d1a-8fbb-da3b27445aa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:18.317552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93412c52-f7f2-4d1a-8fbb-da3b27445aa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:18.326721 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93412c52-f7f2-4d1a-8fbb-da3b27445aa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:18.328201 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93412c52-f7f2-4d1a-8fbb-da3b27445aa6 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:18.337619 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1382/2746] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:15:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:15:18.611721 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae925a5-184f-4914-aec2-b0ffc2c257d0 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:18.614266 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-5ae925a5-184f-4914-aec2-b0ffc2c257d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:18.630276 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae925a5-184f-4914-aec2-b0ffc2c257d0 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:18.635767 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6ad183-d1d4-4db9-86f2-8744fb23ac2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:18.636633 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6ad183-d1d4-4db9-86f2-8744fb23ac2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:18.636915 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6ad183-d1d4-4db9-86f2-8744fb23ac2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:18.637190 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6ad183-d1d4-4db9-86f2-8744fb23ac2b admin admin] PATH_INFO: `/v3/projects/81781a7c2c1a4de694fc4fd498475ba4` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:18.638377 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6ad183-d1d4-4db9-86f2-8744fb23ac2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=81781a7c2c1a4de694fc4fd498475ba4)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:18.640415 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6ad183-d1d4-4db9-86f2-8744fb23ac2b admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:18.687119 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-9d6ad183-d1d4-4db9-86f2-8744fb23ac2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81781a7c2c1a4de694fc4fd498475ba4', 'request_id': 'req-9d6ad183-d1d4-4db9-86f2-8744fb23ac2b'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:18.694478 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-9d6ad183-d1d4-4db9-86f2-8744fb23ac2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81781a7c2c1a4de694fc4fd498475ba4', 'request_id': 'req-9d6ad183-d1d4-4db9-86f2-8744fb23ac2b'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:18.695626 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:INSERT=3,keystone:SELECT=92,keystone:DELETE=5 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:18.702254 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-9d6ad183-d1d4-4db9-86f2-8744fb23ac2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81781a7c2c1a4de694fc4fd498475ba4', 'request_id': 'req-9d6ad183-d1d4-4db9-86f2-8744fb23ac2b'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:18.713824 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-9d6ad183-d1d4-4db9-86f2-8744fb23ac2b admin admin] The token cache is being invalidate because project 81781a7c2c1a4de694fc4fd498475ba4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:15:18.714137 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-9d6ad183-d1d4-4db9-86f2-8744fb23ac2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:18.714771 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-9d6ad183-d1d4-4db9-86f2-8744fb23ac2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:18.716323 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1365/2747] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:15:18 2022] DELETE /identity/v3/projects/81781a7c2c1a4de694fc4fd498475ba4 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:15:18.722895 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031abaa-a584-491f-bd31-a5f693081b57 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:18.723187 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031abaa-a584-491f-bd31-a5f693081b57 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:18.723499 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031abaa-a584-491f-bd31-a5f693081b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:18.742156 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-b031abaa-a584-491f-bd31-a5f693081b57 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:18.743388 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-b031abaa-a584-491f-bd31-a5f693081b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:18.802190 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1383/2748] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:15:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:15:18.809853 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4731aa2a-28eb-4b55-89e3-a967af07148c None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:18.810145 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4731aa2a-28eb-4b55-89e3-a967af07148c None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:18.810426 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4731aa2a-28eb-4b55-89e3-a967af07148c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:18.829250 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-4731aa2a-28eb-4b55-89e3-a967af07148c None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:18.830446 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-4731aa2a-28eb-4b55-89e3-a967af07148c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:18.839227 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1366/2749] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:15:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:15:18.843356 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b031abaa-a584-491f-bd31-a5f693081b57 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:18.853820 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b031abaa-a584-491f-bd31-a5f693081b57 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:18.858301 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f99fc5-1e09-4eeb-8877-a8a4d306aea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:18.858943 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f99fc5-1e09-4eeb-8877-a8a4d306aea5 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:18.859219 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f99fc5-1e09-4eeb-8877-a8a4d306aea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:18.859531 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f99fc5-1e09-4eeb-8877-a8a4d306aea5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:18.860250 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f99fc5-1e09-4eeb-8877-a8a4d306aea5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:15:18.860558 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f99fc5-1e09-4eeb-8877-a8a4d306aea5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:18.862378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f99fc5-1e09-4eeb-8877-a8a4d306aea5 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:18.873888 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1384/2750] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:15:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:15:18.878229 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4731aa2a-28eb-4b55-89e3-a967af07148c None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:18.887445 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4731aa2a-28eb-4b55-89e3-a967af07148c None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:18.892072 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e8434-b9d4-44fb-8e68-abe51aeb690e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:18.892723 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e8434-b9d4-44fb-8e68-abe51aeb690e admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:18.893013 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e8434-b9d4-44fb-8e68-abe51aeb690e admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:18.893831 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e8434-b9d4-44fb-8e68-abe51aeb690e admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:18.894527 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7e8434-b9d4-44fb-8e68-abe51aeb690e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:18.896441 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7e8434-b9d4-44fb-8e68-abe51aeb690e admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:18.921962 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1367/2751] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:15:18 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:15:18.926186 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f99fc5-1e09-4eeb-8877-a8a4d306aea5 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:18.936048 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f99fc5-1e09-4eeb-8877-a8a4d306aea5 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:18.957285 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:18.996955 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86138444-6ce1-4d1e-a2df-0ee7641f130f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:18.997628 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86138444-6ce1-4d1e-a2df-0ee7641f130f admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:18.997912 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86138444-6ce1-4d1e-a2df-0ee7641f130f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:18.998187 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86138444-6ce1-4d1e-a2df-0ee7641f130f admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:18.998880 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86138444-6ce1-4d1e-a2df-0ee7641f130f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:19.000886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86138444-6ce1-4d1e-a2df-0ee7641f130f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:19.020244 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1385/2752] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:15:18 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:15:19.024632 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e8434-b9d4-44fb-8e68-abe51aeb690e admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:19.036594 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e8434-b9d4-44fb-8e68-abe51aeb690e admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:19.041122 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15484ba3-6dd6-4b61-93c0-7a30e56644f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:19.041780 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15484ba3-6dd6-4b61-93c0-7a30e56644f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:19.042063 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15484ba3-6dd6-4b61-93c0-7a30e56644f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:19.042339 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15484ba3-6dd6-4b61-93c0-7a30e56644f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:19.043094 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15484ba3-6dd6-4b61-93c0-7a30e56644f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:19.044707 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15484ba3-6dd6-4b61-93c0-7a30e56644f7 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:19.055918 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1368/2753] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:15:19.059036 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86138444-6ce1-4d1e-a2df-0ee7641f130f admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:19.069428 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86138444-6ce1-4d1e-a2df-0ee7641f130f admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:19.074079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff8d022-78a3-4a8c-bf54-50bd409ef5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:19.074724 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8d022-78a3-4a8c-bf54-50bd409ef5ac admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:19.075006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8d022-78a3-4a8c-bf54-50bd409ef5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:19.075274 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8d022-78a3-4a8c-bf54-50bd409ef5ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:19.076018 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff8d022-78a3-4a8c-bf54-50bd409ef5ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:19.077580 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff8d022-78a3-4a8c-bf54-50bd409ef5ac admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:19.088431 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1386/2754] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:15:19.092670 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15484ba3-6dd6-4b61-93c0-7a30e56644f7 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:19.102940 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15484ba3-6dd6-4b61-93c0-7a30e56644f7 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:19.107740 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaaf56a-39c1-471f-bc94-8735f5989470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:19.108614 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaaf56a-39c1-471f-bc94-8735f5989470 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:19.108895 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaaf56a-39c1-471f-bc94-8735f5989470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:19.109169 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaaf56a-39c1-471f-bc94-8735f5989470 admin admin] PATH_INFO: `/v3/projects/a12ad877f57d4a3d93039417802f31e7/users/5d6588c93a6f498a9163325cae1c7ee8/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:19.123517 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaaf56a-39c1-471f-bc94-8735f5989470 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a12ad877f57d4a3d93039417802f31e7, user_id=5d6588c93a6f498a9163325cae1c7ee8, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:19.126191 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaaf56a-39c1-471f-bc94-8735f5989470 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:19.139841 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1369/2755] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:15:19 2022] PUT /identity/v3/projects/a12ad877f57d4a3d93039417802f31e7/users/5d6588c93a6f498a9163325cae1c7ee8/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:15:19.151999 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4d78a-aa62-4f5d-bd70-90523700b754 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:19.152291 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4d78a-aa62-4f5d-bd70-90523700b754 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:19.152573 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4d78a-aa62-4f5d-bd70-90523700b754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:19.183645 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-56d4d78a-aa62-4f5d-bd70-90523700b754 None None] MFA Rules not processed for user `5d6588c93a6f498a9163325cae1c7ee8`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:19.184846 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-56d4d78a-aa62-4f5d-bd70-90523700b754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:19.247384 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1387/2756] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:15:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:15:19.259173 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaaf56a-39c1-471f-bc94-8735f5989470 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:19.260866 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-faaaf56a-39c1-471f-bc94-8735f5989470 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:19.317435 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaaf56a-39c1-471f-bc94-8735f5989470 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:19.321105 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd5f6fc-a644-4cd4-a629-0dda8d7f907b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:19.321752 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5f6fc-a644-4cd4-a629-0dda8d7f907b service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:19.322038 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5f6fc-a644-4cd4-a629-0dda8d7f907b service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:19.322316 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5f6fc-a644-4cd4-a629-0dda8d7f907b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:19.330544 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd5f6fc-a644-4cd4-a629-0dda8d7f907b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:19.332688 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd5f6fc-a644-4cd4-a629-0dda8d7f907b service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:19.341320 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1370/2757] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:15:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:15:28.165116 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d4d78a-aa62-4f5d-bd70-90523700b754 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:28.278450 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:28.290582 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d4d78a-aa62-4f5d-bd70-90523700b754 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:28.301829 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2261d7fe-c790-4f24-8e13-93618b5d1d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:28.302797 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2261d7fe-c790-4f24-8e13-93618b5d1d11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:28.303150 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2261d7fe-c790-4f24-8e13-93618b5d1d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:28.303524 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2261d7fe-c790-4f24-8e13-93618b5d1d11 admin admin] PATH_INFO: `/v3/users/5d6588c93a6f498a9163325cae1c7ee8` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:28.304838 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2261d7fe-c790-4f24-8e13-93618b5d1d11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d6588c93a6f498a9163325cae1c7ee8)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:28.307130 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2261d7fe-c790-4f24-8e13-93618b5d1d11 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:28.368296 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-2261d7fe-c790-4f24-8e13-93618b5d1d11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d6588c93a6f498a9163325cae1c7ee8'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:28.376004 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-2261d7fe-c790-4f24-8e13-93618b5d1d11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d6588c93a6f498a9163325cae1c7ee8'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:28.399719 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-2261d7fe-c790-4f24-8e13-93618b5d1d11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d6588c93a6f498a9163325cae1c7ee8'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:28.406522 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-2261d7fe-c790-4f24-8e13-93618b5d1d11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d6588c93a6f498a9163325cae1c7ee8'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:28.407796 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-2261d7fe-c790-4f24-8e13-93618b5d1d11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d6588c93a6f498a9163325cae1c7ee8'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:28.432722 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1388/2758] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:15:28 2022] DELETE /identity/v3/users/5d6588c93a6f498a9163325cae1c7ee8 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:15:28.437572 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e965d99b-f567-442d-b7b8-8a11a0547cd6 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:28.438035 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e965d99b-f567-442d-b7b8-8a11a0547cd6 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:28.438119 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e965d99b-f567-442d-b7b8-8a11a0547cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:28.482518 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-e965d99b-f567-442d-b7b8-8a11a0547cd6 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:28.483881 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-e965d99b-f567-442d-b7b8-8a11a0547cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:28.544729 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1371/2759] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:15:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:15:28.556342 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2261d7fe-c790-4f24-8e13-93618b5d1d11 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:28.557828 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-2261d7fe-c790-4f24-8e13-93618b5d1d11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:28.618541 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2261d7fe-c790-4f24-8e13-93618b5d1d11 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:28.622248 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57241ea-292e-4c87-97e2-667ef0f4b371 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:28.623041 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57241ea-292e-4c87-97e2-667ef0f4b371 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:28.623415 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57241ea-292e-4c87-97e2-667ef0f4b371 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:28.623637 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57241ea-292e-4c87-97e2-667ef0f4b371 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:28.633597 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57241ea-292e-4c87-97e2-667ef0f4b371 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:28.635104 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57241ea-292e-4c87-97e2-667ef0f4b371 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:28.645147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1389/2760] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:15:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:15:29.211399 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57241ea-292e-4c87-97e2-667ef0f4b371 service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:29.211978 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e965d99b-f567-442d-b7b8-8a11a0547cd6 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:29.214243 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-e965d99b-f567-442d-b7b8-8a11a0547cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:29.214578 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d57241ea-292e-4c87-97e2-667ef0f4b371 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:29.224191 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:29.266624 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57241ea-292e-4c87-97e2-667ef0f4b371 service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:29.289030 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a807557f-7b94-4049-8f23-61e2c6d1499e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:29.290488 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a807557f-7b94-4049-8f23-61e2c6d1499e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:29.290684 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a807557f-7b94-4049-8f23-61e2c6d1499e admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:29.291087 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a807557f-7b94-4049-8f23-61e2c6d1499e admin admin] PATH_INFO: `/v3/projects/a12ad877f57d4a3d93039417802f31e7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:29.292631 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a807557f-7b94-4049-8f23-61e2c6d1499e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a12ad877f57d4a3d93039417802f31e7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:29.296557 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a807557f-7b94-4049-8f23-61e2c6d1499e admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:29.350685 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e965d99b-f567-442d-b7b8-8a11a0547cd6 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:29.357269 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf8507-905c-4599-a942-da5061198268 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:29.358255 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf8507-905c-4599-a942-da5061198268 service nova] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:29.358255 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf8507-905c-4599-a942-da5061198268 service nova] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:29.358564 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf8507-905c-4599-a942-da5061198268 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:29.361349 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-e1cf8507-905c-4599-a942-da5061198268 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:29.379029 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cf8507-905c-4599-a942-da5061198268 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:29.381212 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cf8507-905c-4599-a942-da5061198268 service nova] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:29.394149 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a807557f-7b94-4049-8f23-61e2c6d1499e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a12ad877f57d4a3d93039417802f31e7', 'request_id': 'req-a807557f-7b94-4049-8f23-61e2c6d1499e'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:29.396295 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1372/2761] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:15:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:15:29.405857 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a807557f-7b94-4049-8f23-61e2c6d1499e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a12ad877f57d4a3d93039417802f31e7', 'request_id': 'req-a807557f-7b94-4049-8f23-61e2c6d1499e'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:29.416568 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a807557f-7b94-4049-8f23-61e2c6d1499e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a12ad877f57d4a3d93039417802f31e7', 'request_id': 'req-a807557f-7b94-4049-8f23-61e2c6d1499e'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:29.420639 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:29.435432 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a807557f-7b94-4049-8f23-61e2c6d1499e admin admin] The token cache is being invalidate because project a12ad877f57d4a3d93039417802f31e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:15:29.436036 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a807557f-7b94-4049-8f23-61e2c6d1499e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:29.436951 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a807557f-7b94-4049-8f23-61e2c6d1499e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:29.438939 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1390/2762] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:15:29 2022] DELETE /identity/v3/projects/a12ad877f57d4a3d93039417802f31e7 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:15:37.007869 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf8507-905c-4599-a942-da5061198268 service nova] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:37.025298 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf8507-905c-4599-a942-da5061198268 service nova] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:37.082994 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c3133-d4f8-4e72-b41d-84af3dca9ad9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:37.083943 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c3133-d4f8-4e72-b41d-84af3dca9ad9 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:37.084229 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c3133-d4f8-4e72-b41d-84af3dca9ad9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:37.084504 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c3133-d4f8-4e72-b41d-84af3dca9ad9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:37.086325 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-2e3c3133-d4f8-4e72-b41d-84af3dca9ad9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:37.150986 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3c3133-d4f8-4e72-b41d-84af3dca9ad9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:37.152453 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3c3133-d4f8-4e72-b41d-84af3dca9ad9 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:37.162529 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1373/2763] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:15:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:15:38.712650 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a807557f-7b94-4049-8f23-61e2c6d1499e admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:38.714093 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a807557f-7b94-4049-8f23-61e2c6d1499e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:38.726655 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a807557f-7b94-4049-8f23-61e2c6d1499e admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:38.732076 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60efcb4-2f08-4dd7-af36-ed8cfd76935c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:38.733025 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60efcb4-2f08-4dd7-af36-ed8cfd76935c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:38.733317 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60efcb4-2f08-4dd7-af36-ed8cfd76935c admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:38.733767 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60efcb4-2f08-4dd7-af36-ed8cfd76935c admin admin] PATH_INFO: `/v3/users/96c957909dc6483f9c8946e70807d791` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:38.735076 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60efcb4-2f08-4dd7-af36-ed8cfd76935c admin admin] RBAC: Authorizing `identity:delete_user(user_id=96c957909dc6483f9c8946e70807d791)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:38.737319 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60efcb4-2f08-4dd7-af36-ed8cfd76935c admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:38.793362 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-b60efcb4-2f08-4dd7-af36-ed8cfd76935c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96c957909dc6483f9c8946e70807d791'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:38.800422 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-b60efcb4-2f08-4dd7-af36-ed8cfd76935c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96c957909dc6483f9c8946e70807d791'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:38.812895 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-b60efcb4-2f08-4dd7-af36-ed8cfd76935c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96c957909dc6483f9c8946e70807d791'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:38.813675 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-b60efcb4-2f08-4dd7-af36-ed8cfd76935c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96c957909dc6483f9c8946e70807d791'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:38.814288 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-b60efcb4-2f08-4dd7-af36-ed8cfd76935c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96c957909dc6483f9c8946e70807d791'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:38.834077 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1391/2764] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:15:38 2022] DELETE /identity/v3/users/96c957909dc6483f9c8946e70807d791 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:15:38.835408 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8b199-8244-42c6-937c-250e4c80eb13 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:38.835813 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8b199-8244-42c6-937c-250e4c80eb13 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:38.836319 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8b199-8244-42c6-937c-250e4c80eb13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:38.865732 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-eef8b199-8244-42c6-937c-250e4c80eb13 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:38.867056 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-eef8b199-8244-42c6-937c-250e4c80eb13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:38.880884 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:38.934889 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1374/2765] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:15:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:15:38.946319 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60efcb4-2f08-4dd7-af36-ed8cfd76935c admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:38.948026 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-b60efcb4-2f08-4dd7-af36-ed8cfd76935c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:39.046187 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60efcb4-2f08-4dd7-af36-ed8cfd76935c admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:39.050191 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d71b54-900e-4cd7-91f2-b7361aa7db11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:39.050916 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d71b54-900e-4cd7-91f2-b7361aa7db11 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:39.051784 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d71b54-900e-4cd7-91f2-b7361aa7db11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:39.052097 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d71b54-900e-4cd7-91f2-b7361aa7db11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:39.066365 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d71b54-900e-4cd7-91f2-b7361aa7db11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:39.068869 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d71b54-900e-4cd7-91f2-b7361aa7db11 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:39.081159 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1392/2766] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:15:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:15:39.404626 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef8b199-8244-42c6-937c-250e4c80eb13 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:39.406123 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-eef8b199-8244-42c6-937c-250e4c80eb13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:39.420974 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef8b199-8244-42c6-937c-250e4c80eb13 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:39.426441 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aafc062-6dda-46ef-afbe-e1ce5eaaf063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:39.428252 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aafc062-6dda-46ef-afbe-e1ce5eaaf063 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:39.428252 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aafc062-6dda-46ef-afbe-e1ce5eaaf063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:39.428252 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aafc062-6dda-46ef-afbe-e1ce5eaaf063 admin admin] PATH_INFO: `/v3/projects/a5f0a610b30645dc98ca47b9581e32b8` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:39.429486 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aafc062-6dda-46ef-afbe-e1ce5eaaf063 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5f0a610b30645dc98ca47b9581e32b8)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:39.431527 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aafc062-6dda-46ef-afbe-e1ce5eaaf063 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:39.484962 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-1aafc062-6dda-46ef-afbe-e1ce5eaaf063 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5f0a610b30645dc98ca47b9581e32b8', 'request_id': 'req-1aafc062-6dda-46ef-afbe-e1ce5eaaf063'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:39.492040 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-1aafc062-6dda-46ef-afbe-e1ce5eaaf063 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5f0a610b30645dc98ca47b9581e32b8', 'request_id': 'req-1aafc062-6dda-46ef-afbe-e1ce5eaaf063'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:39.499822 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-1aafc062-6dda-46ef-afbe-e1ce5eaaf063 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5f0a610b30645dc98ca47b9581e32b8', 'request_id': 'req-1aafc062-6dda-46ef-afbe-e1ce5eaaf063'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:39.512547 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-1aafc062-6dda-46ef-afbe-e1ce5eaaf063 admin admin] The token cache is being invalidate because project a5f0a610b30645dc98ca47b9581e32b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:15:39.512825 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-1aafc062-6dda-46ef-afbe-e1ce5eaaf063 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:39.513497 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-1aafc062-6dda-46ef-afbe-e1ce5eaaf063 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:39.515853 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1375/2767] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:15:39 2022] DELETE /identity/v3/projects/a5f0a610b30645dc98ca47b9581e32b8 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:15:40.578325 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d71b54-900e-4cd7-91f2-b7361aa7db11 service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:40.579925 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-68d71b54-900e-4cd7-91f2-b7361aa7db11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:40.594913 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d71b54-900e-4cd7-91f2-b7361aa7db11 service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:40.647869 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:40.658524 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f90446-6a90-4994-be55-93a0525d0c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:40.659445 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f90446-6a90-4994-be55-93a0525d0c05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:40.659730 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f90446-6a90-4994-be55-93a0525d0c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:40.660006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f90446-6a90-4994-be55-93a0525d0c05 admin admin] PATH_INFO: `/v3/users/de021dc74a4446d8b6a63883fd56a7c4` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:40.675422 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f90446-6a90-4994-be55-93a0525d0c05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de021dc74a4446d8b6a63883fd56a7c4)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:40.677358 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f90446-6a90-4994-be55-93a0525d0c05 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:40.729199 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-65f90446-6a90-4994-be55-93a0525d0c05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de021dc74a4446d8b6a63883fd56a7c4'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:40.735773 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-65f90446-6a90-4994-be55-93a0525d0c05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de021dc74a4446d8b6a63883fd56a7c4'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:40.747604 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-65f90446-6a90-4994-be55-93a0525d0c05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de021dc74a4446d8b6a63883fd56a7c4'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:40.748251 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-65f90446-6a90-4994-be55-93a0525d0c05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de021dc74a4446d8b6a63883fd56a7c4'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:40.748842 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-65f90446-6a90-4994-be55-93a0525d0c05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de021dc74a4446d8b6a63883fd56a7c4'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:40.762934 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1393/2768] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:15:40 2022] DELETE /identity/v3/users/de021dc74a4446d8b6a63883fd56a7c4 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:15:41.019165 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aafc062-6dda-46ef-afbe-e1ce5eaaf063 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:41.021628 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-1aafc062-6dda-46ef-afbe-e1ce5eaaf063 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:41.034501 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aafc062-6dda-46ef-afbe-e1ce5eaaf063 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:41.098738 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420e3996-865e-44bf-8b99-8c7f87d418c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:41.099628 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e3996-865e-44bf-8b99-8c7f87d418c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:41.099908 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e3996-865e-44bf-8b99-8c7f87d418c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:41.100181 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e3996-865e-44bf-8b99-8c7f87d418c6 admin admin] PATH_INFO: `/v3/projects/9f0ee4be50354a20baf5af85aeb4c3b2` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:41.110741 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420e3996-865e-44bf-8b99-8c7f87d418c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f0ee4be50354a20baf5af85aeb4c3b2)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:41.112720 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420e3996-865e-44bf-8b99-8c7f87d418c6 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:41.148238 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:41.158615 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-420e3996-865e-44bf-8b99-8c7f87d418c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f0ee4be50354a20baf5af85aeb4c3b2', 'request_id': 'req-420e3996-865e-44bf-8b99-8c7f87d418c6'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:41.165689 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-420e3996-865e-44bf-8b99-8c7f87d418c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f0ee4be50354a20baf5af85aeb4c3b2', 'request_id': 'req-420e3996-865e-44bf-8b99-8c7f87d418c6'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:41.172099 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-420e3996-865e-44bf-8b99-8c7f87d418c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f0ee4be50354a20baf5af85aeb4c3b2', 'request_id': 'req-420e3996-865e-44bf-8b99-8c7f87d418c6'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:41.182976 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-420e3996-865e-44bf-8b99-8c7f87d418c6 admin admin] The token cache is being invalidate because project 9f0ee4be50354a20baf5af85aeb4c3b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:15:41.183281 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-420e3996-865e-44bf-8b99-8c7f87d418c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:41.183924 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-420e3996-865e-44bf-8b99-8c7f87d418c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:41.185421 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1376/2769] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:15:41 2022] DELETE /identity/v3/projects/9f0ee4be50354a20baf5af85aeb4c3b2 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:15:41.190367 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ebe825-792d-40d7-a8dd-b478598dc4fd None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:41.190663 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ebe825-792d-40d7-a8dd-b478598dc4fd None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:41.190941 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ebe825-792d-40d7-a8dd-b478598dc4fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:41.238395 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-26ebe825-792d-40d7-a8dd-b478598dc4fd None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:41.240125 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-26ebe825-792d-40d7-a8dd-b478598dc4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:41.298045 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1394/2770] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:15:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:15:41.306514 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecde25e-ebf3-47c0-bd02-6e9f2f033583 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:41.306803 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecde25e-ebf3-47c0-bd02-6e9f2f033583 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:41.307087 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecde25e-ebf3-47c0-bd02-6e9f2f033583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:41.325798 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-4ecde25e-ebf3-47c0-bd02-6e9f2f033583 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:41.327008 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-4ecde25e-ebf3-47c0-bd02-6e9f2f033583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:41.335278 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1377/2771] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:15:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:15:41.339486 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ebe825-792d-40d7-a8dd-b478598dc4fd None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:41.349822 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ebe825-792d-40d7-a8dd-b478598dc4fd None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:41.354305 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc01410-f66a-48e4-9425-1b9bedd2c95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:41.354959 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc01410-f66a-48e4-9425-1b9bedd2c95e admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:41.355246 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc01410-f66a-48e4-9425-1b9bedd2c95e admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:41.355582 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc01410-f66a-48e4-9425-1b9bedd2c95e admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:41.356319 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc01410-f66a-48e4-9425-1b9bedd2c95e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:15:41.356635 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc01410-f66a-48e4-9425-1b9bedd2c95e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:41.358245 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc01410-f66a-48e4-9425-1b9bedd2c95e admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:41.370806 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1395/2772] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:15:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:15:41.375097 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecde25e-ebf3-47c0-bd02-6e9f2f033583 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:41.385595 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecde25e-ebf3-47c0-bd02-6e9f2f033583 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:41.390561 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decd4c83-1c1a-44ae-848a-92d83469b8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:41.391221 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd4c83-1c1a-44ae-848a-92d83469b8e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:41.391642 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd4c83-1c1a-44ae-848a-92d83469b8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:41.391930 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd4c83-1c1a-44ae-848a-92d83469b8e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:41.392650 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decd4c83-1c1a-44ae-848a-92d83469b8e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:41.394715 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decd4c83-1c1a-44ae-848a-92d83469b8e7 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:41.427827 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1378/2773] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:15:41 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:15:41.431902 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc01410-f66a-48e4-9425-1b9bedd2c95e admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:41.442609 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc01410-f66a-48e4-9425-1b9bedd2c95e admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:41.462989 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:41.501871 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52085f37-fec1-4997-9fbd-5687203e4112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:41.502531 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52085f37-fec1-4997-9fbd-5687203e4112 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:41.502810 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52085f37-fec1-4997-9fbd-5687203e4112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:41.503078 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52085f37-fec1-4997-9fbd-5687203e4112 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:41.503795 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52085f37-fec1-4997-9fbd-5687203e4112 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:41.506345 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52085f37-fec1-4997-9fbd-5687203e4112 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:41.526270 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1396/2774] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:15:41 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:15:41.530205 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decd4c83-1c1a-44ae-848a-92d83469b8e7 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:41.540722 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decd4c83-1c1a-44ae-848a-92d83469b8e7 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:41.545203 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac9442e-2b7e-48cb-80ae-8e39b8c0b0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:41.545856 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9442e-2b7e-48cb-80ae-8e39b8c0b0d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:41.546136 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9442e-2b7e-48cb-80ae-8e39b8c0b0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:41.546411 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9442e-2b7e-48cb-80ae-8e39b8c0b0d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:41.547137 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac9442e-2b7e-48cb-80ae-8e39b8c0b0d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:41.548742 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac9442e-2b7e-48cb-80ae-8e39b8c0b0d8 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:41.559274 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1379/2775] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:15:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:15:41.562946 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52085f37-fec1-4997-9fbd-5687203e4112 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:41.573148 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52085f37-fec1-4997-9fbd-5687203e4112 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:41.577782 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce6926a-7a57-4186-8a0a-b373c6414f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:41.578433 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce6926a-7a57-4186-8a0a-b373c6414f5f admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:41.578992 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce6926a-7a57-4186-8a0a-b373c6414f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:41.578992 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce6926a-7a57-4186-8a0a-b373c6414f5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:41.579710 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce6926a-7a57-4186-8a0a-b373c6414f5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:41.581285 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce6926a-7a57-4186-8a0a-b373c6414f5f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:41.591268 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1397/2776] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:15:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:15:41.595064 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac9442e-2b7e-48cb-80ae-8e39b8c0b0d8 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:41.605878 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac9442e-2b7e-48cb-80ae-8e39b8c0b0d8 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:41.610884 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3f6c2-881b-4eee-ab86-a152ccc95e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:41.611795 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3f6c2-881b-4eee-ab86-a152ccc95e38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:41.612081 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3f6c2-881b-4eee-ab86-a152ccc95e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:41.612365 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3f6c2-881b-4eee-ab86-a152ccc95e38 admin admin] PATH_INFO: `/v3/projects/7e5fc544bfe8474e84d120073e280797/users/0121a67389b744119e0714acadb33b7f/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:41.627846 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a3f6c2-881b-4eee-ab86-a152ccc95e38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e5fc544bfe8474e84d120073e280797, user_id=0121a67389b744119e0714acadb33b7f, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:41.630598 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a3f6c2-881b-4eee-ab86-a152ccc95e38 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:41.643509 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1380/2777] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:15:41 2022] PUT /identity/v3/projects/7e5fc544bfe8474e84d120073e280797/users/0121a67389b744119e0714acadb33b7f/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:15:41.654244 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10bd9e8-bb09-4530-8150-cf0a1170bf0e None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:41.654540 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10bd9e8-bb09-4530-8150-cf0a1170bf0e None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:41.654849 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10bd9e8-bb09-4530-8150-cf0a1170bf0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:41.682221 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-c10bd9e8-bb09-4530-8150-cf0a1170bf0e None None] MFA Rules not processed for user `0121a67389b744119e0714acadb33b7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:41.683538 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c10bd9e8-bb09-4530-8150-cf0a1170bf0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:41.747210 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1398/2778] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:15:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:15:41.758225 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3f6c2-881b-4eee-ab86-a152ccc95e38 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:41.760321 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-c0a3f6c2-881b-4eee-ab86-a152ccc95e38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:41.837178 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3f6c2-881b-4eee-ab86-a152ccc95e38 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:41.842815 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08081eb6-5abd-437b-8c31-11aec013c1ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:41.843489 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08081eb6-5abd-437b-8c31-11aec013c1ec service nova] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:41.843777 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08081eb6-5abd-437b-8c31-11aec013c1ec service nova] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:41.844053 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08081eb6-5abd-437b-8c31-11aec013c1ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:41.852985 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08081eb6-5abd-437b-8c31-11aec013c1ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:41.855068 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08081eb6-5abd-437b-8c31-11aec013c1ec service nova] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:41.865917 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1381/2779] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:15:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:15:51.291229 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10bd9e8-bb09-4530-8150-cf0a1170bf0e None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:51.356415 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10bd9e8-bb09-4530-8150-cf0a1170bf0e None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:51.361182 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c998e29-26f4-46ce-95ee-2b04422e79fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:51.362608 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c998e29-26f4-46ce-95ee-2b04422e79fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:51.362900 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c998e29-26f4-46ce-95ee-2b04422e79fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:51.363183 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c998e29-26f4-46ce-95ee-2b04422e79fc admin admin] PATH_INFO: `/v3/users/0121a67389b744119e0714acadb33b7f` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:51.365149 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c998e29-26f4-46ce-95ee-2b04422e79fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=0121a67389b744119e0714acadb33b7f)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:51.367449 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c998e29-26f4-46ce-95ee-2b04422e79fc admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:51.403870 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:51.418932 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-0c998e29-26f4-46ce-95ee-2b04422e79fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0121a67389b744119e0714acadb33b7f'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:51.426512 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-0c998e29-26f4-46ce-95ee-2b04422e79fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0121a67389b744119e0714acadb33b7f'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:51.438079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-0c998e29-26f4-46ce-95ee-2b04422e79fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0121a67389b744119e0714acadb33b7f'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:51.438760 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-0c998e29-26f4-46ce-95ee-2b04422e79fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0121a67389b744119e0714acadb33b7f'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:51.439379 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-0c998e29-26f4-46ce-95ee-2b04422e79fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0121a67389b744119e0714acadb33b7f'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:51.452191 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1399/2780] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:15:51 2022] DELETE /identity/v3/users/0121a67389b744119e0714acadb33b7f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:15:51.456781 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cab9e50-8de3-4e36-bfd7-4e0e0e437b2c None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:51.457401 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cab9e50-8de3-4e36-bfd7-4e0e0e437b2c None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:51.457570 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cab9e50-8de3-4e36-bfd7-4e0e0e437b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:51.474875 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-6cab9e50-8de3-4e36-bfd7-4e0e0e437b2c None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:51.476133 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-6cab9e50-8de3-4e36-bfd7-4e0e0e437b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:51.517429 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:51.538778 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1382/2781] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:15:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:15:51.548419 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c998e29-26f4-46ce-95ee-2b04422e79fc admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:51.549816 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-0c998e29-26f4-46ce-95ee-2b04422e79fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:51.610371 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c998e29-26f4-46ce-95ee-2b04422e79fc admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:51.615676 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe0e22e-d368-4f47-bfd8-3b22163e4eb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:51.616448 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0e22e-d368-4f47-bfd8-3b22163e4eb7 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:51.616755 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0e22e-d368-4f47-bfd8-3b22163e4eb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:51.617210 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0e22e-d368-4f47-bfd8-3b22163e4eb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:51.627163 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe0e22e-d368-4f47-bfd8-3b22163e4eb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:51.628619 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe0e22e-d368-4f47-bfd8-3b22163e4eb7 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:51.638456 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1400/2782] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:15:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:15:51.802428 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd84758d-84c5-4fdf-9f92-8b79c192770b None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:51.802730 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd84758d-84c5-4fdf-9f92-8b79c192770b None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:51.803017 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd84758d-84c5-4fdf-9f92-8b79c192770b None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:51.804722 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1383/2783] 10.222.0.13 () {54 vars in 921 bytes} [Tue Jun 7 22:15:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 22:15:51.812278 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f1339a-bb9c-4ac3-8d8b-56f48bd86498 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:51.812587 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f1339a-bb9c-4ac3-8d8b-56f48bd86498 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:51.812941 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f1339a-bb9c-4ac3-8d8b-56f48bd86498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:51.852979 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-30f1339a-bb9c-4ac3-8d8b-56f48bd86498 None None] MFA Rules not processed for user `10242251e6284217bfb79c67b1567f75`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:51.854845 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-30f1339a-bb9c-4ac3-8d8b-56f48bd86498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:51.863194 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1401/2784] 10.222.0.13 () {58 vars in 1018 bytes} [Tue Jun 7 22:15:51 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:15:51.895226 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd84758d-84c5-4fdf-9f92-8b79c192770b None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:51.910986 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd84758d-84c5-4fdf-9f92-8b79c192770b None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:51.915630 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76659650-4728-4edb-9b32-8fd26b3cf2c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:51.916357 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76659650-4728-4edb-9b32-8fd26b3cf2c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:51.916711 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76659650-4728-4edb-9b32-8fd26b3cf2c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:51.917062 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76659650-4728-4edb-9b32-8fd26b3cf2c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:51.925501 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76659650-4728-4edb-9b32-8fd26b3cf2c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:51.927660 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76659650-4728-4edb-9b32-8fd26b3cf2c7 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:51.940925 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1384/2785] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:15:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:15:52.023829 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f1339a-bb9c-4ac3-8d8b-56f48bd86498 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:52.025582 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-30f1339a-bb9c-4ac3-8d8b-56f48bd86498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:52.057510 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f1339a-bb9c-4ac3-8d8b-56f48bd86498 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:52.058755 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f0e3dc-5f56-4607-917e-752d24c6dd31 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:52.058976 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f0e3dc-5f56-4607-917e-752d24c6dd31 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:52.059479 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f0e3dc-5f56-4607-917e-752d24c6dd31 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:52.061820 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1385/2786] 10.222.0.13 () {54 vars in 921 bytes} [Tue Jun 7 22:15:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 22:15:52.064902 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c691ca-30a0-4bcf-9703-b2c00376e460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:52.068613 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1bb1e4-71ab-4275-b3ed-19595bd28c8e None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:52.068906 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1bb1e4-71ab-4275-b3ed-19595bd28c8e None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:52.069186 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1bb1e4-71ab-4275-b3ed-19595bd28c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:52.071235 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c691ca-30a0-4bcf-9703-b2c00376e460 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:52.071559 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c691ca-30a0-4bcf-9703-b2c00376e460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:52.071840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c691ca-30a0-4bcf-9703-b2c00376e460 admin admin] PATH_INFO: `/v3/projects/7e5fc544bfe8474e84d120073e280797` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:52.073319 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c691ca-30a0-4bcf-9703-b2c00376e460 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e5fc544bfe8474e84d120073e280797)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:52.075239 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c691ca-30a0-4bcf-9703-b2c00376e460 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:52.089480 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-fc1bb1e4-71ab-4275-b3ed-19595bd28c8e None None] MFA Rules not processed for user `dbe0303ea0764347bfb054870ebba41f`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:52.090694 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-fc1bb1e4-71ab-4275-b3ed-19595bd28c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:52.134938 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c8c691ca-30a0-4bcf-9703-b2c00376e460 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e5fc544bfe8474e84d120073e280797', 'request_id': 'req-c8c691ca-30a0-4bcf-9703-b2c00376e460'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:52.145468 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c8c691ca-30a0-4bcf-9703-b2c00376e460 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e5fc544bfe8474e84d120073e280797', 'request_id': 'req-c8c691ca-30a0-4bcf-9703-b2c00376e460'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:52.152310 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c8c691ca-30a0-4bcf-9703-b2c00376e460 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e5fc544bfe8474e84d120073e280797', 'request_id': 'req-c8c691ca-30a0-4bcf-9703-b2c00376e460'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:52.164886 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c8c691ca-30a0-4bcf-9703-b2c00376e460 admin admin] The token cache is being invalidate because project 7e5fc544bfe8474e84d120073e280797 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:15:52.165266 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c8c691ca-30a0-4bcf-9703-b2c00376e460 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:52.165935 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c8c691ca-30a0-4bcf-9703-b2c00376e460 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:15:52.170973 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1402/2787] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:15:52 2022] DELETE /identity/v3/projects/7e5fc544bfe8474e84d120073e280797 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:15:52.172482 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2d656-81a7-4b79-bd0f-4b20e51261f5 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:52.172769 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2d656-81a7-4b79-bd0f-4b20e51261f5 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:52.173049 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2d656-81a7-4b79-bd0f-4b20e51261f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:52.182950 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1386/2788] 10.222.0.13 () {58 vars in 1018 bytes} [Tue Jun 7 22:15:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:15:52.189889 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1bb1e4-71ab-4275-b3ed-19595bd28c8e None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:52.190771 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-fc1bb1e4-71ab-4275-b3ed-19595bd28c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:52.193840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-68e2d656-81a7-4b79-bd0f-4b20e51261f5 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:52.194954 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-68e2d656-81a7-4b79-bd0f-4b20e51261f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:52.202921 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1bb1e4-71ab-4275-b3ed-19595bd28c8e None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:52.204760 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:52.253140 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b27ab0-56e0-4f9f-ae5d-3acd10e29c1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:52.254342 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b27ab0-56e0-4f9f-ae5d-3acd10e29c1e service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:52.254795 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b27ab0-56e0-4f9f-ae5d-3acd10e29c1e service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:52.255125 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b27ab0-56e0-4f9f-ae5d-3acd10e29c1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:52.257773 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1403/2789] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:15:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:15:52.265336 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1870917e-1d67-48a7-8e9a-aad609214669 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:52.265694 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1870917e-1d67-48a7-8e9a-aad609214669 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:52.266128 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1870917e-1d67-48a7-8e9a-aad609214669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:52.284449 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-1870917e-1d67-48a7-8e9a-aad609214669 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:52.286012 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-1870917e-1d67-48a7-8e9a-aad609214669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:52.294355 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1404/2790] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:15:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:15:52.298309 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1870917e-1d67-48a7-8e9a-aad609214669 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:52.307813 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1870917e-1d67-48a7-8e9a-aad609214669 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:52.312676 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13111a38-39fd-4388-96cc-6cd12eebfb01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:52.313415 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13111a38-39fd-4388-96cc-6cd12eebfb01 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:52.313710 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13111a38-39fd-4388-96cc-6cd12eebfb01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:52.313977 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13111a38-39fd-4388-96cc-6cd12eebfb01 admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:52.314704 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13111a38-39fd-4388-96cc-6cd12eebfb01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:15:52.315015 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13111a38-39fd-4388-96cc-6cd12eebfb01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:52.317199 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13111a38-39fd-4388-96cc-6cd12eebfb01 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:52.330824 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b27ab0-56e0-4f9f-ae5d-3acd10e29c1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:52.336370 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b27ab0-56e0-4f9f-ae5d-3acd10e29c1e service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:52.337911 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1405/2791] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:15:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:15:52.341810 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13111a38-39fd-4388-96cc-6cd12eebfb01 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:52.347164 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1387/2792] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:15:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 22:15:52.351799 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13111a38-39fd-4388-96cc-6cd12eebfb01 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:52.356826 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad832c52-738c-40b2-b629-fb2062f23f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:52.357485 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad832c52-738c-40b2-b629-fb2062f23f85 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:52.357788 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad832c52-738c-40b2-b629-fb2062f23f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:52.358073 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad832c52-738c-40b2-b629-fb2062f23f85 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:52.360348 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad832c52-738c-40b2-b629-fb2062f23f85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:52.360705 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad832c52-738c-40b2-b629-fb2062f23f85 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:52.395012 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1406/2793] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:15:52 2022] POST /identity/v3/projects => generated 365 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:15:52.399019 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b27ab0-56e0-4f9f-ae5d-3acd10e29c1e service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:52.406040 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:52.411255 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b27ab0-56e0-4f9f-ae5d-3acd10e29c1e service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:52.418203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad832c52-738c-40b2-b629-fb2062f23f85 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:52.420617 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-ad832c52-738c-40b2-b629-fb2062f23f85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:52.479843 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910e32a8-b538-418b-ab85-4c10fdb4eae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:52.480633 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e32a8-b538-418b-ab85-4c10fdb4eae2 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:52.480916 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e32a8-b538-418b-ab85-4c10fdb4eae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:52.481192 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e32a8-b538-418b-ab85-4c10fdb4eae2 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:52.481890 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910e32a8-b538-418b-ab85-4c10fdb4eae2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:52.483836 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910e32a8-b538-418b-ab85-4c10fdb4eae2 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:52.500401 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad832c52-738c-40b2-b629-fb2062f23f85 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:52.506073 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0942e-66eb-45a4-b00a-e5f7a304ca3f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02941c6ad2d54e48b1114d9e030c66c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:52.506726 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0942e-66eb-45a4-b00a-e5f7a304ca3f service placement] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:52.507007 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0942e-66eb-45a4-b00a-e5f7a304ca3f service placement] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:52.507282 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0942e-66eb-45a4-b00a-e5f7a304ca3f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:52.507686 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1388/2794] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:15:52 2022] POST /identity/v3/users => generated 283 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:15:52.508583 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910e32a8-b538-418b-ab85-4c10fdb4eae2 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:52.509099 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-84a0942e-66eb-45a4-b00a-e5f7a304ca3f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:52.517866 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910e32a8-b538-418b-ab85-4c10fdb4eae2 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:52.521688 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a0942e-66eb-45a4-b00a-e5f7a304ca3f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:52.523271 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a0942e-66eb-45a4-b00a-e5f7a304ca3f service placement] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:52.523412 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8fcb06-c435-4a58-8092-5def1fabb947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:52.523951 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fcb06-c435-4a58-8092-5def1fabb947 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:52.524233 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fcb06-c435-4a58-8092-5def1fabb947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:52.524508 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fcb06-c435-4a58-8092-5def1fabb947 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:52.525233 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8fcb06-c435-4a58-8092-5def1fabb947 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:52.526791 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8fcb06-c435-4a58-8092-5def1fabb947 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:52.533526 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1407/2795] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:15:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 22:15:52.538366 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1389/2796] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 22:15:52.542211 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0942e-66eb-45a4-b00a-e5f7a304ca3f service placement] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:52.552414 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0942e-66eb-45a4-b00a-e5f7a304ca3f service placement] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:52.556998 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb65f2db-cac2-4efa-84f6-cb8d26943f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:52.557645 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb65f2db-cac2-4efa-84f6-cb8d26943f08 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:52.557932 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb65f2db-cac2-4efa-84f6-cb8d26943f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:52.558212 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb65f2db-cac2-4efa-84f6-cb8d26943f08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:52.558939 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb65f2db-cac2-4efa-84f6-cb8d26943f08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:52.560607 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb65f2db-cac2-4efa-84f6-cb8d26943f08 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:52.571584 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1408/2797] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:15:52.575845 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8fcb06-c435-4a58-8092-5def1fabb947 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:52.586819 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8fcb06-c435-4a58-8092-5def1fabb947 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:52.591759 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e6403-cb06-44f7-8f8d-c110b2629390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:52.592676 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e6403-cb06-44f7-8f8d-c110b2629390 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:52.592974 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e6403-cb06-44f7-8f8d-c110b2629390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:52.593253 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e6403-cb06-44f7-8f8d-c110b2629390 admin admin] PATH_INFO: `/v3/projects/40068272b81d485a930d76801cca11f6/users/d7109536a7b64d518ae2021b3f56e6f9/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:52.611610 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e6403-cb06-44f7-8f8d-c110b2629390 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40068272b81d485a930d76801cca11f6, user_id=d7109536a7b64d518ae2021b3f56e6f9, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:52.614827 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e6403-cb06-44f7-8f8d-c110b2629390 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:52.628643 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1390/2798] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:15:52 2022] PUT /identity/v3/projects/40068272b81d485a930d76801cca11f6/users/d7109536a7b64d518ae2021b3f56e6f9/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:15:52.640251 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac36f32-b2eb-4367-a432-ea0a523662cd None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:52.640544 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac36f32-b2eb-4367-a432-ea0a523662cd None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:52.640824 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac36f32-b2eb-4367-a432-ea0a523662cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:52.670275 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-eac36f32-b2eb-4367-a432-ea0a523662cd None None] MFA Rules not processed for user `d7109536a7b64d518ae2021b3f56e6f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:15:52.671525 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-eac36f32-b2eb-4367-a432-ea0a523662cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:52.718319 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:15:52.749755 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1409/2799] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:15:52 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:15:52.758198 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e6403-cb06-44f7-8f8d-c110b2629390 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:52.762116 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-f65e6403-cb06-44f7-8f8d-c110b2629390 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:52.832853 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e6403-cb06-44f7-8f8d-c110b2629390 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:52.836815 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a40a7d-dfdf-4922-9b6b-7a027cd3df89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:52.837514 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a40a7d-dfdf-4922-9b6b-7a027cd3df89 service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:52.837800 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a40a7d-dfdf-4922-9b6b-7a027cd3df89 service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:52.838084 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a40a7d-dfdf-4922-9b6b-7a027cd3df89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:52.847973 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a40a7d-dfdf-4922-9b6b-7a027cd3df89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:52.849529 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a40a7d-dfdf-4922-9b6b-7a027cd3df89 service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:52.860209 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1391/2800] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:15:52 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:15:53.478158 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac36f32-b2eb-4367-a432-ea0a523662cd None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:15:53.487809 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac36f32-b2eb-4367-a432-ea0a523662cd None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:15:53.492378 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9789c3f-096e-48ec-94f1-c30026c656cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:15:53.493007 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9789c3f-096e-48ec-94f1-c30026c656cd service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:15:53.493285 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9789c3f-096e-48ec-94f1-c30026c656cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:15:53.493563 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9789c3f-096e-48ec-94f1-c30026c656cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:15:53.495411 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-f9789c3f-096e-48ec-94f1-c30026c656cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:15:53.574271 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9789c3f-096e-48ec-94f1-c30026c656cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:15:53.575660 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9789c3f-096e-48ec-94f1-c30026c656cd service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:15:53.593603 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1410/2801] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:15:53 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:16:01.294127 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a40a7d-dfdf-4922-9b6b-7a027cd3df89 service cinder] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:16:01.308182 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9789c3f-096e-48ec-94f1-c30026c656cd service cinder] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:16:01.355567 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a40a7d-dfdf-4922-9b6b-7a027cd3df89 service cinder] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:16:01.359376 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d285d8e-a9fd-4f21-a832-ff2bfc74fe1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:16:01.360021 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d285d8e-a9fd-4f21-a832-ff2bfc74fe1c service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:01.360304 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d285d8e-a9fd-4f21-a832-ff2bfc74fe1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:01.360690 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d285d8e-a9fd-4f21-a832-ff2bfc74fe1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:01.362391 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-4d285d8e-a9fd-4f21-a832-ff2bfc74fe1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:16:01.373236 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9789c3f-096e-48ec-94f1-c30026c656cd service cinder] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:16:01.374629 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d285d8e-a9fd-4f21-a832-ff2bfc74fe1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:16:01.376098 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d285d8e-a9fd-4f21-a832-ff2bfc74fe1c service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:16:01.377820 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446029a2-dd12-468c-9630-9ae9c3490169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:16:01.378712 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446029a2-dd12-468c-9630-9ae9c3490169 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:01.379046 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446029a2-dd12-468c-9630-9ae9c3490169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:01.379266 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446029a2-dd12-468c-9630-9ae9c3490169 admin admin] PATH_INFO: `/v3/users/d7109536a7b64d518ae2021b3f56e6f9` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:01.380967 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446029a2-dd12-468c-9630-9ae9c3490169 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7109536a7b64d518ae2021b3f56e6f9)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:16:01.382869 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446029a2-dd12-468c-9630-9ae9c3490169 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:16:01.385780 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1392/2802] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:16:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:16:01.439592 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-446029a2-dd12-468c-9630-9ae9c3490169 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7109536a7b64d518ae2021b3f56e6f9'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:01.446843 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:16:01.449015 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-446029a2-dd12-468c-9630-9ae9c3490169 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7109536a7b64d518ae2021b3f56e6f9'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:01.463457 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-446029a2-dd12-468c-9630-9ae9c3490169 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7109536a7b64d518ae2021b3f56e6f9'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:01.464184 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-446029a2-dd12-468c-9630-9ae9c3490169 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7109536a7b64d518ae2021b3f56e6f9'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:01.464819 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-446029a2-dd12-468c-9630-9ae9c3490169 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7109536a7b64d518ae2021b3f56e6f9'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:01.479099 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1411/2803] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:16:01 2022] DELETE /identity/v3/users/d7109536a7b64d518ae2021b3f56e6f9 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:16:01.483544 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ea7db-bbe1-4160-9a9f-1e74f6e6740e None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:01.483839 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ea7db-bbe1-4160-9a9f-1e74f6e6740e None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:01.484171 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ea7db-bbe1-4160-9a9f-1e74f6e6740e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:01.491245 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:16:01.502851 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-796ea7db-bbe1-4160-9a9f-1e74f6e6740e None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:16:01.504197 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-796ea7db-bbe1-4160-9a9f-1e74f6e6740e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:16:01.566486 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1393/2804] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:16:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:16:01.577351 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446029a2-dd12-468c-9630-9ae9c3490169 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:16:01.578896 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-446029a2-dd12-468c-9630-9ae9c3490169 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:16:01.649279 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446029a2-dd12-468c-9630-9ae9c3490169 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:16:01.654416 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa18d66-7b07-47c6-aad2-21fd947a1686 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:16:01.655077 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa18d66-7b07-47c6-aad2-21fd947a1686 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:01.655386 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa18d66-7b07-47c6-aad2-21fd947a1686 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:01.655665 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa18d66-7b07-47c6-aad2-21fd947a1686 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:01.666471 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa18d66-7b07-47c6-aad2-21fd947a1686 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:16:01.668147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa18d66-7b07-47c6-aad2-21fd947a1686 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:16:01.681084 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1412/2805] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:16:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:16:01.967894 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ea7db-bbe1-4160-9a9f-1e74f6e6740e None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:16:01.969354 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-796ea7db-bbe1-4160-9a9f-1e74f6e6740e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:16:01.986267 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ea7db-bbe1-4160-9a9f-1e74f6e6740e None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:16:01.993561 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e51e07-4d39-4d16-bd41-977536af5b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:16:01.994449 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e51e07-4d39-4d16-bd41-977536af5b77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:01.994731 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e51e07-4d39-4d16-bd41-977536af5b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:01.995054 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e51e07-4d39-4d16-bd41-977536af5b77 admin admin] PATH_INFO: `/v3/projects/40068272b81d485a930d76801cca11f6` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:01.996983 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e51e07-4d39-4d16-bd41-977536af5b77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40068272b81d485a930d76801cca11f6)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:16:01.998914 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e51e07-4d39-4d16-bd41-977536af5b77 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:16:02.050285 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-60e51e07-4d39-4d16-bd41-977536af5b77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40068272b81d485a930d76801cca11f6', 'request_id': 'req-60e51e07-4d39-4d16-bd41-977536af5b77'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:02.059204 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-60e51e07-4d39-4d16-bd41-977536af5b77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40068272b81d485a930d76801cca11f6', 'request_id': 'req-60e51e07-4d39-4d16-bd41-977536af5b77'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:02.065679 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-60e51e07-4d39-4d16-bd41-977536af5b77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40068272b81d485a930d76801cca11f6', 'request_id': 'req-60e51e07-4d39-4d16-bd41-977536af5b77'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:02.076949 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-60e51e07-4d39-4d16-bd41-977536af5b77 admin admin] The token cache is being invalidate because project 40068272b81d485a930d76801cca11f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:16:02.077242 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-60e51e07-4d39-4d16-bd41-977536af5b77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:02.077885 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-60e51e07-4d39-4d16-bd41-977536af5b77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:02.079453 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1394/2806] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:16:01 2022] DELETE /identity/v3/projects/40068272b81d485a930d76801cca11f6 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:16:02.085557 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ceb251-449a-48e5-ad5a-e03ccb5c59ea None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:02.085841 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ceb251-449a-48e5-ad5a-e03ccb5c59ea None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:02.086115 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ceb251-449a-48e5-ad5a-e03ccb5c59ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:02.104529 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-90ceb251-449a-48e5-ad5a-e03ccb5c59ea None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:16:02.105726 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-90ceb251-449a-48e5-ad5a-e03ccb5c59ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:16:02.168361 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1413/2807] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:16:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:16:02.176214 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bafc60-eea7-4477-9b28-c2fed5a20729 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:02.176600 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bafc60-eea7-4477-9b28-c2fed5a20729 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:02.177449 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bafc60-eea7-4477-9b28-c2fed5a20729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:02.196592 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-45bafc60-eea7-4477-9b28-c2fed5a20729 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:16:02.197904 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-45bafc60-eea7-4477-9b28-c2fed5a20729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:16:02.205737 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1395/2808] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:16:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:16:02.210251 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ceb251-449a-48e5-ad5a-e03ccb5c59ea None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:16:02.219969 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ceb251-449a-48e5-ad5a-e03ccb5c59ea None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:16:02.224783 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc656303-c8c1-46a5-ae12-5e97c0027ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:16:02.225515 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc656303-c8c1-46a5-ae12-5e97c0027ee3 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:02.225862 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc656303-c8c1-46a5-ae12-5e97c0027ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:02.226228 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc656303-c8c1-46a5-ae12-5e97c0027ee3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:02.227102 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc656303-c8c1-46a5-ae12-5e97c0027ee3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:16:02.227518 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc656303-c8c1-46a5-ae12-5e97c0027ee3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:16:02.229433 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc656303-c8c1-46a5-ae12-5e97c0027ee3 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:16:02.241729 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1414/2809] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:16:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:16:02.246071 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bafc60-eea7-4477-9b28-c2fed5a20729 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:16:02.259961 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bafc60-eea7-4477-9b28-c2fed5a20729 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:16:02.266207 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f2e47-721e-436f-b320-636e4d279de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:16:02.267418 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f2e47-721e-436f-b320-636e4d279de1 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:02.267957 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f2e47-721e-436f-b320-636e4d279de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:02.268468 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f2e47-721e-436f-b320-636e4d279de1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:02.269273 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508f2e47-721e-436f-b320-636e4d279de1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:16:02.271275 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508f2e47-721e-436f-b320-636e4d279de1 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:16:02.300827 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1396/2810] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:16:02 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:16:02.301916 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc656303-c8c1-46a5-ae12-5e97c0027ee3 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:16:02.312851 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc656303-c8c1-46a5-ae12-5e97c0027ee3 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:16:02.382762 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88dd648-7d26-4626-9d6c-8d5448c88bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:16:02.383482 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88dd648-7d26-4626-9d6c-8d5448c88bca admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:02.383768 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88dd648-7d26-4626-9d6c-8d5448c88bca admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:02.384044 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88dd648-7d26-4626-9d6c-8d5448c88bca admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:02.384761 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88dd648-7d26-4626-9d6c-8d5448c88bca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:16:02.386720 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88dd648-7d26-4626-9d6c-8d5448c88bca admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:16:02.396404 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:16:02.415006 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1415/2811] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:16:02 2022] POST /identity/v3/users => generated 287 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:16:02.419348 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f2e47-721e-436f-b320-636e4d279de1 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:16:02.430498 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f2e47-721e-436f-b320-636e4d279de1 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:16:02.435988 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a1a61-4f6a-4daf-b8d7-d3ee922a982c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:16:02.436670 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a1a61-4f6a-4daf-b8d7-d3ee922a982c admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:02.437006 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a1a61-4f6a-4daf-b8d7-d3ee922a982c admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:02.437287 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a1a61-4f6a-4daf-b8d7-d3ee922a982c admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:02.438063 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5a1a61-4f6a-4daf-b8d7-d3ee922a982c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:16:02.439747 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5a1a61-4f6a-4daf-b8d7-d3ee922a982c admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:16:02.451933 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1397/2812] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:16:02.455887 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88dd648-7d26-4626-9d6c-8d5448c88bca admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:16:02.467222 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88dd648-7d26-4626-9d6c-8d5448c88bca admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:16:02.473747 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e84b130-c7c6-49b8-8323-ff91398af8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:16:02.475253 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84b130-c7c6-49b8-8323-ff91398af8ac admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:02.475631 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84b130-c7c6-49b8-8323-ff91398af8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:02.475966 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84b130-c7c6-49b8-8323-ff91398af8ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:02.476770 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e84b130-c7c6-49b8-8323-ff91398af8ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:16:02.478756 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e84b130-c7c6-49b8-8323-ff91398af8ac admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:16:02.490592 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1416/2813] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 22:16:02.494757 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a1a61-4f6a-4daf-b8d7-d3ee922a982c admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:16:02.508052 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a1a61-4f6a-4daf-b8d7-d3ee922a982c admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:16:02.514301 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c27ba-9439-4523-8249-72f351946fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:16:02.514720 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c27ba-9439-4523-8249-72f351946fe0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:02.515109 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c27ba-9439-4523-8249-72f351946fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:02.515529 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c27ba-9439-4523-8249-72f351946fe0 admin admin] PATH_INFO: `/v3/projects/f322e8741dea4ac79507613619cf492f/users/2337187672394b768594057290e57f02/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:02.534235 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97c27ba-9439-4523-8249-72f351946fe0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f322e8741dea4ac79507613619cf492f, user_id=2337187672394b768594057290e57f02, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:16:02.537459 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97c27ba-9439-4523-8249-72f351946fe0 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:16:02.551280 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1398/2814] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:16:02 2022] PUT /identity/v3/projects/f322e8741dea4ac79507613619cf492f/users/2337187672394b768594057290e57f02/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:16:02.562527 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c7a53-a525-4310-a62b-b82fb0d4bfbc None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:02.562820 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c7a53-a525-4310-a62b-b82fb0d4bfbc None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:02.563096 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c7a53-a525-4310-a62b-b82fb0d4bfbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:02.589659 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-226c7a53-a525-4310-a62b-b82fb0d4bfbc None None] MFA Rules not processed for user `2337187672394b768594057290e57f02`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:16:02.590876 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-226c7a53-a525-4310-a62b-b82fb0d4bfbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:16:02.661036 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1417/2815] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:16:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:16:02.666323 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c27ba-9439-4523-8249-72f351946fe0 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:16:02.695672 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:16:02.737770 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c27ba-9439-4523-8249-72f351946fe0 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:16:02.744266 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb617995-463c-4f38-bb54-ddcabd3a13ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:16:02.744410 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb617995-463c-4f38-bb54-ddcabd3a13ee admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:02.744587 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb617995-463c-4f38-bb54-ddcabd3a13ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:02.744865 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb617995-463c-4f38-bb54-ddcabd3a13ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:02.745576 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb617995-463c-4f38-bb54-ddcabd3a13ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:16:02.747562 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb617995-463c-4f38-bb54-ddcabd3a13ee admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:16:02.782201 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1399/2816] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:16:02 2022] POST /identity/v3/projects => generated 371 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:16:02.783100 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c7a53-a525-4310-a62b-b82fb0d4bfbc None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:16:02.799276 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c7a53-a525-4310-a62b-b82fb0d4bfbc None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:16:02.871412 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76812c4-d9e6-4e0c-a457-5245b7c2e8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:16:02.872744 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76812c4-d9e6-4e0c-a457-5245b7c2e8cb admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:02.873126 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76812c4-d9e6-4e0c-a457-5245b7c2e8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:02.874510 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76812c4-d9e6-4e0c-a457-5245b7c2e8cb admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:02.874510 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76812c4-d9e6-4e0c-a457-5245b7c2e8cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:16:02.876131 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76812c4-d9e6-4e0c-a457-5245b7c2e8cb admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:16:02.901901 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1418/2817] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:16:02 2022] POST /identity/v3/users => generated 292 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:16:02.903606 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb617995-463c-4f38-bb54-ddcabd3a13ee admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:16:02.915645 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb617995-463c-4f38-bb54-ddcabd3a13ee admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:16:02.922297 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4978a-28d6-44f5-837e-9e7cebd2c31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:16:02.923089 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4978a-28d6-44f5-837e-9e7cebd2c31e admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:02.923486 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4978a-28d6-44f5-837e-9e7cebd2c31e admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:02.923956 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4978a-28d6-44f5-837e-9e7cebd2c31e admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:02.925848 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd4978a-28d6-44f5-837e-9e7cebd2c31e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:16:02.926757 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd4978a-28d6-44f5-837e-9e7cebd2c31e admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:16:02.939486 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1400/2818] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:16:02.944194 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76812c4-d9e6-4e0c-a457-5245b7c2e8cb admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:16:02.954645 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76812c4-d9e6-4e0c-a457-5245b7c2e8cb admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:16:02.960803 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38d276-4856-4790-bbc4-07d78669bdab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:16:02.961779 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38d276-4856-4790-bbc4-07d78669bdab admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:02.962143 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38d276-4856-4790-bbc4-07d78669bdab admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:02.962501 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38d276-4856-4790-bbc4-07d78669bdab admin admin] PATH_INFO: `/v3/projects/ee84214d6e3a4631b76929e48cc5dbbb/users/11286104dbc94ca095a9573fd8a659e0/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:02.979663 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e38d276-4856-4790-bbc4-07d78669bdab admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee84214d6e3a4631b76929e48cc5dbbb, user_id=11286104dbc94ca095a9573fd8a659e0, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:16:02.982538 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e38d276-4856-4790-bbc4-07d78669bdab admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:16:02.997879 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1419/2819] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:16:02 2022] PUT /identity/v3/projects/ee84214d6e3a4631b76929e48cc5dbbb/users/11286104dbc94ca095a9573fd8a659e0/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:16:03.001687 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4978a-28d6-44f5-837e-9e7cebd2c31e admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:16:03.013388 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4978a-28d6-44f5-837e-9e7cebd2c31e admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:16:03.085302 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917620ed-ab07-423b-9eb7-ddd9fee59db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:16:03.086378 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917620ed-ab07-423b-9eb7-ddd9fee59db4 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:03.086757 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917620ed-ab07-423b-9eb7-ddd9fee59db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:03.087110 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917620ed-ab07-423b-9eb7-ddd9fee59db4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:03.087966 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917620ed-ab07-423b-9eb7-ddd9fee59db4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:16:03.089644 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917620ed-ab07-423b-9eb7-ddd9fee59db4 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:16:03.106871 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1401/2820] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:16:03.108839 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38d276-4856-4790-bbc4-07d78669bdab admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:16:03.122910 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38d276-4856-4790-bbc4-07d78669bdab admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:16:03.129872 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f25c55d-4569-4788-99c5-e6ab94fef3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:16:03.130812 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f25c55d-4569-4788-99c5-e6ab94fef3a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:03.131095 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f25c55d-4569-4788-99c5-e6ab94fef3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:03.131400 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f25c55d-4569-4788-99c5-e6ab94fef3a3 admin admin] PATH_INFO: `/v3/projects/ee84214d6e3a4631b76929e48cc5dbbb/users/11286104dbc94ca095a9573fd8a659e0/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:03.134076 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f25c55d-4569-4788-99c5-e6ab94fef3a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee84214d6e3a4631b76929e48cc5dbbb, user_id=11286104dbc94ca095a9573fd8a659e0, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:16:03.137336 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f25c55d-4569-4788-99c5-e6ab94fef3a3 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:16:03.163509 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1420/2821] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:16:03 2022] PUT /identity/v3/projects/ee84214d6e3a4631b76929e48cc5dbbb/users/11286104dbc94ca095a9573fd8a659e0/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:16:03.164823 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4182933f-0075-4229-aeb9-3ea35413d23d None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:03.165113 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4182933f-0075-4229-aeb9-3ea35413d23d None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:03.165388 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4182933f-0075-4229-aeb9-3ea35413d23d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:03.193801 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-4182933f-0075-4229-aeb9-3ea35413d23d None None] MFA Rules not processed for user `11286104dbc94ca095a9573fd8a659e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:16:03.195040 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-4182933f-0075-4229-aeb9-3ea35413d23d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:16:03.279134 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1402/2822] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:16:03 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:16:03.306147 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f25c55d-4569-4788-99c5-e6ab94fef3a3 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:16:03.308101 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-9f25c55d-4569-4788-99c5-e6ab94fef3a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:16:03.340763 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:16:03.388338 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f25c55d-4569-4788-99c5-e6ab94fef3a3 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:16:03.392103 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867d2c47-654d-40e1-ab6f-918e96d61204 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:16:03.393135 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d2c47-654d-40e1-ab6f-918e96d61204 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:03.393135 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d2c47-654d-40e1-ab6f-918e96d61204 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:03.393350 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d2c47-654d-40e1-ab6f-918e96d61204 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:03.404680 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867d2c47-654d-40e1-ab6f-918e96d61204 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:16:03.406127 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867d2c47-654d-40e1-ab6f-918e96d61204 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:16:03.418846 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1421/2823] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:16:03 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:16:03.506245 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4182933f-0075-4229-aeb9-3ea35413d23d None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:16:03.518262 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4182933f-0075-4229-aeb9-3ea35413d23d None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:16:03.529843 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8b8038-1723-4113-952d-aa43ea075969 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:16:03.530721 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b8038-1723-4113-952d-aa43ea075969 service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:03.531086 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b8038-1723-4113-952d-aa43ea075969 service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:03.531500 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b8038-1723-4113-952d-aa43ea075969 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:03.556723 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:16:03.605682 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8b8038-1723-4113-952d-aa43ea075969 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:16:03.607218 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8b8038-1723-4113-952d-aa43ea075969 service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:16:03.619081 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1403/2824] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:16:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:16:08.221519 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867d2c47-654d-40e1-ab6f-918e96d61204 service cinder] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:16:08.223257 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-867d2c47-654d-40e1-ab6f-918e96d61204 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:16:08.290330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867d2c47-654d-40e1-ab6f-918e96d61204 service cinder] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:16:08.294813 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebfd6de-ad34-452a-8622-24e5cf736984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:16:08.295712 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebfd6de-ad34-452a-8622-24e5cf736984 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:08.295990 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebfd6de-ad34-452a-8622-24e5cf736984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:08.296259 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebfd6de-ad34-452a-8622-24e5cf736984 admin admin] PATH_INFO: `/v3/users/c710942802df49ec8bd630e4b225eed2` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:08.297455 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebfd6de-ad34-452a-8622-24e5cf736984 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c710942802df49ec8bd630e4b225eed2)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:16:08.299407 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebfd6de-ad34-452a-8622-24e5cf736984 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:16:08.346679 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-1ebfd6de-ad34-452a-8622-24e5cf736984 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c710942802df49ec8bd630e4b225eed2'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:08.354103 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-1ebfd6de-ad34-452a-8622-24e5cf736984 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c710942802df49ec8bd630e4b225eed2'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:08.367384 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-1ebfd6de-ad34-452a-8622-24e5cf736984 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c710942802df49ec8bd630e4b225eed2'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:08.368003 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-1ebfd6de-ad34-452a-8622-24e5cf736984 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c710942802df49ec8bd630e4b225eed2'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:08.368597 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-1ebfd6de-ad34-452a-8622-24e5cf736984 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c710942802df49ec8bd630e4b225eed2'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:08.382368 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1422/2825] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:16:08 2022] DELETE /identity/v3/users/c710942802df49ec8bd630e4b225eed2 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:16:08.556828 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8b8038-1723-4113-952d-aa43ea075969 service cinder] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:16:08.569400 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8b8038-1723-4113-952d-aa43ea075969 service cinder] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:16:08.628967 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48b8e8f-e13b-445d-9cf1-17d16d16628d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:16:08.629712 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b8e8f-e13b-445d-9cf1-17d16d16628d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:08.630044 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b8e8f-e13b-445d-9cf1-17d16d16628d admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:08.630319 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b8e8f-e13b-445d-9cf1-17d16d16628d admin admin] PATH_INFO: `/v3/projects/579eb78d8af74e2cadce3a0c793b4440` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:08.631539 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48b8e8f-e13b-445d-9cf1-17d16d16628d admin admin] RBAC: Authorizing `identity:delete_project(project_id=579eb78d8af74e2cadce3a0c793b4440)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:16:08.633407 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48b8e8f-e13b-445d-9cf1-17d16d16628d admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:16:08.680131 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-e48b8e8f-e13b-445d-9cf1-17d16d16628d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '579eb78d8af74e2cadce3a0c793b4440', 'request_id': 'req-e48b8e8f-e13b-445d-9cf1-17d16d16628d'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:08.686938 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-e48b8e8f-e13b-445d-9cf1-17d16d16628d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '579eb78d8af74e2cadce3a0c793b4440', 'request_id': 'req-e48b8e8f-e13b-445d-9cf1-17d16d16628d'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:08.693608 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-e48b8e8f-e13b-445d-9cf1-17d16d16628d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '579eb78d8af74e2cadce3a0c793b4440', 'request_id': 'req-e48b8e8f-e13b-445d-9cf1-17d16d16628d'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:08.703662 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-e48b8e8f-e13b-445d-9cf1-17d16d16628d admin admin] The token cache is being invalidate because project 579eb78d8af74e2cadce3a0c793b4440 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:16:08.703944 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-e48b8e8f-e13b-445d-9cf1-17d16d16628d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:08.704571 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-e48b8e8f-e13b-445d-9cf1-17d16d16628d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:08.706076 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1404/2826] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:16:08 2022] DELETE /identity/v3/projects/579eb78d8af74e2cadce3a0c793b4440 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:16:08.710706 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebfd6de-ad34-452a-8622-24e5cf736984 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:16:08.712198 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-1ebfd6de-ad34-452a-8622-24e5cf736984 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:16:08.725901 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebfd6de-ad34-452a-8622-24e5cf736984 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:16:08.771552 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:16:08.787946 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f402a2-bdb9-4d94-9027-dce85118ce6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:16:08.788823 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f402a2-bdb9-4d94-9027-dce85118ce6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:08.789108 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f402a2-bdb9-4d94-9027-dce85118ce6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:08.789386 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f402a2-bdb9-4d94-9027-dce85118ce6d admin admin] PATH_INFO: `/v3/users/61973960102c4341889b29825fb38940` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:08.803056 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f402a2-bdb9-4d94-9027-dce85118ce6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=61973960102c4341889b29825fb38940)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:16:08.805019 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f402a2-bdb9-4d94-9027-dce85118ce6d admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:16:08.848438 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-07f402a2-bdb9-4d94-9027-dce85118ce6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61973960102c4341889b29825fb38940'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:08.856017 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-07f402a2-bdb9-4d94-9027-dce85118ce6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61973960102c4341889b29825fb38940'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:08.868440 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-07f402a2-bdb9-4d94-9027-dce85118ce6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61973960102c4341889b29825fb38940'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:08.868816 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-07f402a2-bdb9-4d94-9027-dce85118ce6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61973960102c4341889b29825fb38940'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:08.869333 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-07f402a2-bdb9-4d94-9027-dce85118ce6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61973960102c4341889b29825fb38940'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:08.882506 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1423/2827] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:16:08 2022] DELETE /identity/v3/users/61973960102c4341889b29825fb38940 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:16:09.082737 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48b8e8f-e13b-445d-9cf1-17d16d16628d admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:16:09.084746 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-e48b8e8f-e13b-445d-9cf1-17d16d16628d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:16:09.100474 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48b8e8f-e13b-445d-9cf1-17d16d16628d admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:16:09.147661 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:16:09.161166 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7509ff9-7be0-4611-bfa1-269a092e52dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:16:09.162613 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7509ff9-7be0-4611-bfa1-269a092e52dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:09.162891 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7509ff9-7be0-4611-bfa1-269a092e52dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:09.163162 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7509ff9-7be0-4611-bfa1-269a092e52dc admin admin] PATH_INFO: `/v3/projects/0a1a5d620d604e468fcf86e5cdd09467` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:09.173925 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7509ff9-7be0-4611-bfa1-269a092e52dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a1a5d620d604e468fcf86e5cdd09467)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:16:09.175875 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7509ff9-7be0-4611-bfa1-269a092e52dc admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:16:09.221254 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-f7509ff9-7be0-4611-bfa1-269a092e52dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a1a5d620d604e468fcf86e5cdd09467', 'request_id': 'req-f7509ff9-7be0-4611-bfa1-269a092e52dc'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:09.228192 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-f7509ff9-7be0-4611-bfa1-269a092e52dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a1a5d620d604e468fcf86e5cdd09467', 'request_id': 'req-f7509ff9-7be0-4611-bfa1-269a092e52dc'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:09.235182 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-f7509ff9-7be0-4611-bfa1-269a092e52dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a1a5d620d604e468fcf86e5cdd09467', 'request_id': 'req-f7509ff9-7be0-4611-bfa1-269a092e52dc'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:09.245882 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-f7509ff9-7be0-4611-bfa1-269a092e52dc admin admin] The token cache is being invalidate because project 0a1a5d620d604e468fcf86e5cdd09467 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:16:09.246171 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-f7509ff9-7be0-4611-bfa1-269a092e52dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:09.246818 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-f7509ff9-7be0-4611-bfa1-269a092e52dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:09.248417 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1405/2828] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:16:09 2022] DELETE /identity/v3/projects/0a1a5d620d604e468fcf86e5cdd09467 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:16:18.878712 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=38,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:16:19.242873 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=33,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:16:47.283441 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f402a2-bdb9-4d94-9027-dce85118ce6d admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:16:47.285075 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-07f402a2-bdb9-4d94-9027-dce85118ce6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:16:47.299575 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f402a2-bdb9-4d94-9027-dce85118ce6d admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:16:47.357454 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32365e35-d1aa-4a33-9621-c9b5fcfbd0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:16:47.358327 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32365e35-d1aa-4a33-9621-c9b5fcfbd0e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:47.358602 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32365e35-d1aa-4a33-9621-c9b5fcfbd0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:47.358873 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32365e35-d1aa-4a33-9621-c9b5fcfbd0e9 admin admin] PATH_INFO: `/v3/users/2337187672394b768594057290e57f02` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:47.360202 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32365e35-d1aa-4a33-9621-c9b5fcfbd0e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2337187672394b768594057290e57f02)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:16:47.362145 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32365e35-d1aa-4a33-9621-c9b5fcfbd0e9 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:16:47.408481 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-32365e35-d1aa-4a33-9621-c9b5fcfbd0e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2337187672394b768594057290e57f02'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:47.415208 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-32365e35-d1aa-4a33-9621-c9b5fcfbd0e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2337187672394b768594057290e57f02'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:47.426607 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-32365e35-d1aa-4a33-9621-c9b5fcfbd0e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2337187672394b768594057290e57f02'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:47.427287 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-32365e35-d1aa-4a33-9621-c9b5fcfbd0e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2337187672394b768594057290e57f02'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:47.427917 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-32365e35-d1aa-4a33-9621-c9b5fcfbd0e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2337187672394b768594057290e57f02'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:47.440330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1424/2829] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:16:47 2022] DELETE /identity/v3/users/2337187672394b768594057290e57f02 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:16:47.445293 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b702a1-4ea0-48c5-badb-673151634d9a None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:47.446346 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b702a1-4ea0-48c5-badb-673151634d9a None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:47.446346 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b702a1-4ea0-48c5-badb-673151634d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:47.462820 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-13b702a1-4ea0-48c5-badb-673151634d9a None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:16:47.464028 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-13b702a1-4ea0-48c5-badb-673151634d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:16:47.521496 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1406/2830] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:16:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:16:47.531622 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32365e35-d1aa-4a33-9621-c9b5fcfbd0e9 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:16:47.533062 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-32365e35-d1aa-4a33-9621-c9b5fcfbd0e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:16:47.594824 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32365e35-d1aa-4a33-9621-c9b5fcfbd0e9 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:16:47.598491 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6267b6-a104-42d6-9677-2dcf1cbe6cc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:16:47.599132 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6267b6-a104-42d6-9677-2dcf1cbe6cc8 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:47.599458 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6267b6-a104-42d6-9677-2dcf1cbe6cc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:47.599734 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6267b6-a104-42d6-9677-2dcf1cbe6cc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:47.609241 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6267b6-a104-42d6-9677-2dcf1cbe6cc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:16:47.610666 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6267b6-a104-42d6-9677-2dcf1cbe6cc8 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:16:47.621160 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1425/2831] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:16:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:16:47.901109 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b702a1-4ea0-48c5-badb-673151634d9a None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:16:47.902601 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-13b702a1-4ea0-48c5-badb-673151634d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:16:47.914780 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b702a1-4ea0-48c5-badb-673151634d9a None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:16:47.920343 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97537d98-58a0-47f4-9fd1-32577e3f4cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:16:47.921395 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97537d98-58a0-47f4-9fd1-32577e3f4cb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:47.921688 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97537d98-58a0-47f4-9fd1-32577e3f4cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:47.921966 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97537d98-58a0-47f4-9fd1-32577e3f4cb2 admin admin] PATH_INFO: `/v3/projects/f322e8741dea4ac79507613619cf492f` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:47.923275 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97537d98-58a0-47f4-9fd1-32577e3f4cb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f322e8741dea4ac79507613619cf492f)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:16:47.926083 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97537d98-58a0-47f4-9fd1-32577e3f4cb2 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:16:47.980588 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-97537d98-58a0-47f4-9fd1-32577e3f4cb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f322e8741dea4ac79507613619cf492f', 'request_id': 'req-97537d98-58a0-47f4-9fd1-32577e3f4cb2'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:47.988676 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-97537d98-58a0-47f4-9fd1-32577e3f4cb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f322e8741dea4ac79507613619cf492f', 'request_id': 'req-97537d98-58a0-47f4-9fd1-32577e3f4cb2'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:47.995581 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-97537d98-58a0-47f4-9fd1-32577e3f4cb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f322e8741dea4ac79507613619cf492f', 'request_id': 'req-97537d98-58a0-47f4-9fd1-32577e3f4cb2'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:48.005838 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-97537d98-58a0-47f4-9fd1-32577e3f4cb2 admin admin] The token cache is being invalidate because project f322e8741dea4ac79507613619cf492f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:16:48.006129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-97537d98-58a0-47f4-9fd1-32577e3f4cb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:48.006772 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-97537d98-58a0-47f4-9fd1-32577e3f4cb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:48.008883 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1407/2832] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:16:47 2022] DELETE /identity/v3/projects/f322e8741dea4ac79507613619cf492f => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:16:48.012885 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6267b6-a104-42d6-9677-2dcf1cbe6cc8 service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:16:48.022360 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6267b6-a104-42d6-9677-2dcf1cbe6cc8 service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:16:48.040452 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:16:48.081005 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc28838-932c-49b9-bbd9-4057b7a98c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:16:48.081879 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc28838-932c-49b9-bbd9-4057b7a98c0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:48.082162 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc28838-932c-49b9-bbd9-4057b7a98c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:48.082437 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc28838-932c-49b9-bbd9-4057b7a98c0c admin admin] PATH_INFO: `/v3/users/11286104dbc94ca095a9573fd8a659e0` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:48.083657 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc28838-932c-49b9-bbd9-4057b7a98c0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=11286104dbc94ca095a9573fd8a659e0)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:16:48.085696 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc28838-932c-49b9-bbd9-4057b7a98c0c admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:16:48.133800 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8fc28838-932c-49b9-bbd9-4057b7a98c0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11286104dbc94ca095a9573fd8a659e0'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:48.140586 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8fc28838-932c-49b9-bbd9-4057b7a98c0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11286104dbc94ca095a9573fd8a659e0'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:48.152286 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8fc28838-932c-49b9-bbd9-4057b7a98c0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11286104dbc94ca095a9573fd8a659e0'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:48.152964 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8fc28838-932c-49b9-bbd9-4057b7a98c0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11286104dbc94ca095a9573fd8a659e0'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:48.153571 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8fc28838-932c-49b9-bbd9-4057b7a98c0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11286104dbc94ca095a9573fd8a659e0'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:48.166218 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1426/2833] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:16:48 2022] DELETE /identity/v3/users/11286104dbc94ca095a9573fd8a659e0 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:16:48.441658 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97537d98-58a0-47f4-9fd1-32577e3f4cb2 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:16:48.443120 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-97537d98-58a0-47f4-9fd1-32577e3f4cb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:16:48.456979 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97537d98-58a0-47f4-9fd1-32577e3f4cb2 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:16:48.514563 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dd61b6-dc91-421a-a765-a9f44d9e3855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:16:48.515459 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd61b6-dc91-421a-a765-a9f44d9e3855 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:48.515747 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd61b6-dc91-421a-a765-a9f44d9e3855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:48.516051 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd61b6-dc91-421a-a765-a9f44d9e3855 admin admin] PATH_INFO: `/v3/projects/ee84214d6e3a4631b76929e48cc5dbbb` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:48.517246 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dd61b6-dc91-421a-a765-a9f44d9e3855 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee84214d6e3a4631b76929e48cc5dbbb)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:16:48.519138 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dd61b6-dc91-421a-a765-a9f44d9e3855 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:16:48.551484 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:16:48.568662 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-17dd61b6-dc91-421a-a765-a9f44d9e3855 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee84214d6e3a4631b76929e48cc5dbbb', 'request_id': 'req-17dd61b6-dc91-421a-a765-a9f44d9e3855'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:48.575836 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-17dd61b6-dc91-421a-a765-a9f44d9e3855 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee84214d6e3a4631b76929e48cc5dbbb', 'request_id': 'req-17dd61b6-dc91-421a-a765-a9f44d9e3855'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:48.581838 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-17dd61b6-dc91-421a-a765-a9f44d9e3855 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee84214d6e3a4631b76929e48cc5dbbb', 'request_id': 'req-17dd61b6-dc91-421a-a765-a9f44d9e3855'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:48.593020 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-17dd61b6-dc91-421a-a765-a9f44d9e3855 admin admin] The token cache is being invalidate because project ee84214d6e3a4631b76929e48cc5dbbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:16:48.593303 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-17dd61b6-dc91-421a-a765-a9f44d9e3855 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:48.593934 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-17dd61b6-dc91-421a-a765-a9f44d9e3855 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:48.595496 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1408/2834] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:16:48 2022] DELETE /identity/v3/projects/ee84214d6e3a4631b76929e48cc5dbbb => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:16:48.600287 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c62112-4a04-411b-9b67-23be43475950 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:48.600799 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c62112-4a04-411b-9b67-23be43475950 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:48.600949 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c62112-4a04-411b-9b67-23be43475950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:48.621403 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-12c62112-4a04-411b-9b67-23be43475950 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:16:48.622596 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-12c62112-4a04-411b-9b67-23be43475950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:16:48.681993 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1427/2835] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:16:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:16:48.689681 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11ff6b-202d-4190-9893-3ecd6238e4e7 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:48.689975 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11ff6b-202d-4190-9893-3ecd6238e4e7 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:48.690261 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11ff6b-202d-4190-9893-3ecd6238e4e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:48.708739 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-cd11ff6b-202d-4190-9893-3ecd6238e4e7 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:16:48.709936 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-cd11ff6b-202d-4190-9893-3ecd6238e4e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:16:48.718288 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1409/2836] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:16:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 22:16:48.722296 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c62112-4a04-411b-9b67-23be43475950 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:16:48.732984 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c62112-4a04-411b-9b67-23be43475950 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:16:48.737942 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f4c91-3d15-46a7-a56c-7aedd530559e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:16:48.738568 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f4c91-3d15-46a7-a56c-7aedd530559e admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:48.738967 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f4c91-3d15-46a7-a56c-7aedd530559e admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:48.739366 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f4c91-3d15-46a7-a56c-7aedd530559e admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:48.740005 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f4c91-3d15-46a7-a56c-7aedd530559e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:16:48.740302 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f4c91-3d15-46a7-a56c-7aedd530559e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:16:48.742063 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f4c91-3d15-46a7-a56c-7aedd530559e admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:16:48.754087 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1428/2837] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:16:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:16:48.758618 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd11ff6b-202d-4190-9893-3ecd6238e4e7 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:16:48.767890 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd11ff6b-202d-4190-9893-3ecd6238e4e7 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:16:48.772524 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b30e69-654e-4823-b340-626c8fafae82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:16:48.773674 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b30e69-654e-4823-b340-626c8fafae82 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:48.773979 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b30e69-654e-4823-b340-626c8fafae82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:48.774279 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b30e69-654e-4823-b340-626c8fafae82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:48.775054 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b30e69-654e-4823-b340-626c8fafae82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:16:48.777013 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b30e69-654e-4823-b340-626c8fafae82 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:16:48.802620 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1410/2838] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:16:48 2022] POST /identity/v3/projects => generated 381 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 22:16:48.806881 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f4c91-3d15-46a7-a56c-7aedd530559e admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:16:48.816756 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f4c91-3d15-46a7-a56c-7aedd530559e admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:16:48.842497 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:16:48.875469 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aa7210-ca76-4593-a59b-14db5c9cb8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:16:48.876133 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aa7210-ca76-4593-a59b-14db5c9cb8b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:48.876416 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aa7210-ca76-4593-a59b-14db5c9cb8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:48.876689 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aa7210-ca76-4593-a59b-14db5c9cb8b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:48.877371 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aa7210-ca76-4593-a59b-14db5c9cb8b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:16:48.879268 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aa7210-ca76-4593-a59b-14db5c9cb8b6 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:16:48.898983 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1429/2839] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:16:48 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:16:48.903128 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b30e69-654e-4823-b340-626c8fafae82 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:16:48.913320 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b30e69-654e-4823-b340-626c8fafae82 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:16:48.918179 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b7e36b-b56c-4c44-83d1-6486245cd3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:16:48.918863 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b7e36b-b56c-4c44-83d1-6486245cd3d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:48.919150 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b7e36b-b56c-4c44-83d1-6486245cd3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:48.919466 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b7e36b-b56c-4c44-83d1-6486245cd3d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:48.920200 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b7e36b-b56c-4c44-83d1-6486245cd3d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:16:48.921809 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b7e36b-b56c-4c44-83d1-6486245cd3d5 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:16:48.932164 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1411/2840] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:16:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:16:48.935977 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aa7210-ca76-4593-a59b-14db5c9cb8b6 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:16:48.946402 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aa7210-ca76-4593-a59b-14db5c9cb8b6 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:16:48.951002 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec95a584-dbf0-47d0-869b-d246a8544e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:16:48.951685 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec95a584-dbf0-47d0-869b-d246a8544e84 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:48.951971 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec95a584-dbf0-47d0-869b-d246a8544e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:48.952252 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec95a584-dbf0-47d0-869b-d246a8544e84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:48.952979 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec95a584-dbf0-47d0-869b-d246a8544e84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:16:48.954562 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec95a584-dbf0-47d0-869b-d246a8544e84 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:16:48.964733 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1430/2841] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:16:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:16:48.969027 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b7e36b-b56c-4c44-83d1-6486245cd3d5 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:16:48.981872 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b7e36b-b56c-4c44-83d1-6486245cd3d5 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:16:48.989009 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d34d614-3f6e-434b-b4fc-ff0f140efd7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:16:48.990378 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34d614-3f6e-434b-b4fc-ff0f140efd7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:48.990807 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34d614-3f6e-434b-b4fc-ff0f140efd7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:48.991269 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34d614-3f6e-434b-b4fc-ff0f140efd7c admin admin] PATH_INFO: `/v3/projects/25954e1508c340f1b78c3753ae9ac2db/users/fc5ac9d0f824427ea9969ef1cf45a6ce/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:49.009025 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d34d614-3f6e-434b-b4fc-ff0f140efd7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=25954e1508c340f1b78c3753ae9ac2db, user_id=fc5ac9d0f824427ea9969ef1cf45a6ce, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:16:49.011694 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d34d614-3f6e-434b-b4fc-ff0f140efd7c admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:16:49.024695 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1412/2842] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:16:48 2022] PUT /identity/v3/projects/25954e1508c340f1b78c3753ae9ac2db/users/fc5ac9d0f824427ea9969ef1cf45a6ce/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:16:49.035675 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5db322c-ed46-40d6-80b1-0fe430b1aaa1 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:49.035964 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5db322c-ed46-40d6-80b1-0fe430b1aaa1 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:49.036239 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5db322c-ed46-40d6-80b1-0fe430b1aaa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:49.063490 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-a5db322c-ed46-40d6-80b1-0fe430b1aaa1 None None] MFA Rules not processed for user `fc5ac9d0f824427ea9969ef1cf45a6ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:16:49.064683 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a5db322c-ed46-40d6-80b1-0fe430b1aaa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:16:49.128097 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1431/2843] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:16:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:16:49.132669 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d34d614-3f6e-434b-b4fc-ff0f140efd7c admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:16:49.192771 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d34d614-3f6e-434b-b4fc-ff0f140efd7c admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:16:49.197655 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620fb4ca-293b-466d-9a3b-329e4aa837d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:16:49.198298 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620fb4ca-293b-466d-9a3b-329e4aa837d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:49.198577 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620fb4ca-293b-466d-9a3b-329e4aa837d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:49.198852 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620fb4ca-293b-466d-9a3b-329e4aa837d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:49.199569 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620fb4ca-293b-466d-9a3b-329e4aa837d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:16:49.201429 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620fb4ca-293b-466d-9a3b-329e4aa837d3 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:16:49.225949 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1413/2844] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:16:49 2022] POST /identity/v3/projects => generated 381 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:16:49.230900 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5db322c-ed46-40d6-80b1-0fe430b1aaa1 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:16:49.241330 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5db322c-ed46-40d6-80b1-0fe430b1aaa1 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:16:49.304926 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f072589-fb31-405c-9a27-28a294ed559c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:16:49.305683 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f072589-fb31-405c-9a27-28a294ed559c admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:49.305964 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f072589-fb31-405c-9a27-28a294ed559c admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:49.306239 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f072589-fb31-405c-9a27-28a294ed559c admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:49.306934 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f072589-fb31-405c-9a27-28a294ed559c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:16:49.309465 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f072589-fb31-405c-9a27-28a294ed559c admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:16:49.329674 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1432/2845] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:16:49 2022] POST /identity/v3/users => generated 297 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:16:49.333674 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620fb4ca-293b-466d-9a3b-329e4aa837d3 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:16:49.345105 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620fb4ca-293b-466d-9a3b-329e4aa837d3 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:16:49.349682 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1106205-6498-4673-b372-e7b0dda33e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:16:49.350341 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1106205-6498-4673-b372-e7b0dda33e1b admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:49.350626 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1106205-6498-4673-b372-e7b0dda33e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:49.350906 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1106205-6498-4673-b372-e7b0dda33e1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:49.352324 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1106205-6498-4673-b372-e7b0dda33e1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:16:49.354142 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1106205-6498-4673-b372-e7b0dda33e1b admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:16:49.364640 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1414/2846] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:16:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:16:49.368783 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f072589-fb31-405c-9a27-28a294ed559c admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:16:49.378905 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f072589-fb31-405c-9a27-28a294ed559c admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:16:49.383553 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ff4b38-63ce-47f1-b8b0-8885f592ac8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:16:49.384487 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff4b38-63ce-47f1-b8b0-8885f592ac8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:49.384773 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff4b38-63ce-47f1-b8b0-8885f592ac8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:49.385044 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff4b38-63ce-47f1-b8b0-8885f592ac8b admin admin] PATH_INFO: `/v3/projects/fe784a6f03d34bd1a6f7e7b933ab5c89/users/972c5991960645de80141535724998be/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:49.400515 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ff4b38-63ce-47f1-b8b0-8885f592ac8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe784a6f03d34bd1a6f7e7b933ab5c89, user_id=972c5991960645de80141535724998be, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:16:49.403204 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ff4b38-63ce-47f1-b8b0-8885f592ac8b admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:16:49.416602 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1433/2847] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:16:49 2022] PUT /identity/v3/projects/fe784a6f03d34bd1a6f7e7b933ab5c89/users/972c5991960645de80141535724998be/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:16:49.420499 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1106205-6498-4673-b372-e7b0dda33e1b admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:16:49.430443 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1106205-6498-4673-b372-e7b0dda33e1b admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:16:49.449621 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:16:49.488466 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e96aea-42d3-42ea-8f27-4f3fce3d4ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:16:49.489116 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e96aea-42d3-42ea-8f27-4f3fce3d4ba7 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:49.489393 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e96aea-42d3-42ea-8f27-4f3fce3d4ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:49.489669 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e96aea-42d3-42ea-8f27-4f3fce3d4ba7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:49.490380 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e96aea-42d3-42ea-8f27-4f3fce3d4ba7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:16:49.492067 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e96aea-42d3-42ea-8f27-4f3fce3d4ba7 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:16:49.502059 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1415/2848] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:16:49 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:16:49.506185 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ff4b38-63ce-47f1-b8b0-8885f592ac8b admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:16:49.511853 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:16:49.517969 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ff4b38-63ce-47f1-b8b0-8885f592ac8b admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:16:49.523695 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7af741-29ef-45be-816f-f6e6d0308325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:16:49.524683 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7af741-29ef-45be-816f-f6e6d0308325 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:49.525079 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7af741-29ef-45be-816f-f6e6d0308325 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:49.525352 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7af741-29ef-45be-816f-f6e6d0308325 admin admin] PATH_INFO: `/v3/projects/fe784a6f03d34bd1a6f7e7b933ab5c89/users/972c5991960645de80141535724998be/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:49.527357 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7af741-29ef-45be-816f-f6e6d0308325 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe784a6f03d34bd1a6f7e7b933ab5c89, user_id=972c5991960645de80141535724998be, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:16:49.529952 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7af741-29ef-45be-816f-f6e6d0308325 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:16:49.540666 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1434/2849] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:16:49 2022] PUT /identity/v3/projects/fe784a6f03d34bd1a6f7e7b933ab5c89/users/972c5991960645de80141535724998be/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:16:49.550668 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329075e5-91e4-4835-b91f-2662817352a2 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:49.550961 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329075e5-91e4-4835-b91f-2662817352a2 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:49.551242 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329075e5-91e4-4835-b91f-2662817352a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:49.576727 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-329075e5-91e4-4835-b91f-2662817352a2 None None] MFA Rules not processed for user `972c5991960645de80141535724998be`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:16:49.577908 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-329075e5-91e4-4835-b91f-2662817352a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:16:49.646387 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1416/2850] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:16:49 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:16:49.658641 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7af741-29ef-45be-816f-f6e6d0308325 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:16:49.660453 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-8e7af741-29ef-45be-816f-f6e6d0308325 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:16:49.717936 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7af741-29ef-45be-816f-f6e6d0308325 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:16:49.721728 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fec757-76bf-41c3-824c-b891c440c34d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:16:49.722365 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fec757-76bf-41c3-824c-b891c440c34d service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:49.722648 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fec757-76bf-41c3-824c-b891c440c34d service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:49.722923 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fec757-76bf-41c3-824c-b891c440c34d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:49.732433 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fec757-76bf-41c3-824c-b891c440c34d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:16:49.733857 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fec757-76bf-41c3-824c-b891c440c34d service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:16:49.743695 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1435/2851] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:16:49 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:16:49.812895 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329075e5-91e4-4835-b91f-2662817352a2 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:16:49.823316 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329075e5-91e4-4835-b91f-2662817352a2 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:16:49.826948 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f33270-168d-4f05-93c0-18a0e8126d75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:16:49.827630 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f33270-168d-4f05-93c0-18a0e8126d75 service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:49.827916 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f33270-168d-4f05-93c0-18a0e8126d75 service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:49.828192 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f33270-168d-4f05-93c0-18a0e8126d75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:49.884713 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f33270-168d-4f05-93c0-18a0e8126d75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:16:49.886140 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f33270-168d-4f05-93c0-18a0e8126d75 service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:16:49.895257 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1417/2852] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:16:49 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:16:59.676446 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fec757-76bf-41c3-824c-b891c440c34d service cinder] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:16:59.738165 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fec757-76bf-41c3-824c-b891c440c34d service cinder] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:16:59.743527 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804fa5e9-e7bd-41cc-a079-841f77985e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:16:59.744382 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804fa5e9-e7bd-41cc-a079-841f77985e67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:59.744664 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804fa5e9-e7bd-41cc-a079-841f77985e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:59.744962 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804fa5e9-e7bd-41cc-a079-841f77985e67 admin admin] PATH_INFO: `/v3/users/fc5ac9d0f824427ea9969ef1cf45a6ce` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:59.746177 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804fa5e9-e7bd-41cc-a079-841f77985e67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc5ac9d0f824427ea9969ef1cf45a6ce)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:16:59.748129 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804fa5e9-e7bd-41cc-a079-841f77985e67 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:16:59.794259 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-804fa5e9-e7bd-41cc-a079-841f77985e67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc5ac9d0f824427ea9969ef1cf45a6ce'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:59.801182 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-804fa5e9-e7bd-41cc-a079-841f77985e67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc5ac9d0f824427ea9969ef1cf45a6ce'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:59.813196 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-804fa5e9-e7bd-41cc-a079-841f77985e67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc5ac9d0f824427ea9969ef1cf45a6ce'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:59.813890 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-804fa5e9-e7bd-41cc-a079-841f77985e67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc5ac9d0f824427ea9969ef1cf45a6ce'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:59.814492 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-804fa5e9-e7bd-41cc-a079-841f77985e67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc5ac9d0f824427ea9969ef1cf45a6ce'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:16:59.827767 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1436/2853] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:16:59 2022] DELETE /identity/v3/users/fc5ac9d0f824427ea9969ef1cf45a6ce => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:16:59.832357 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcbe628-296b-471e-b2d2-f372855be32f None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:59.832878 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcbe628-296b-471e-b2d2-f372855be32f None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:59.832878 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcbe628-296b-471e-b2d2-f372855be32f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:59.849929 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-1bcbe628-296b-471e-b2d2-f372855be32f None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:16:59.851748 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-1bcbe628-296b-471e-b2d2-f372855be32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:16:59.892704 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:16:59.911891 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1418/2854] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:16:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:16:59.921791 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804fa5e9-e7bd-41cc-a079-841f77985e67 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:16:59.923205 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-804fa5e9-e7bd-41cc-a079-841f77985e67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:16:59.937074 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:16:59.981588 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804fa5e9-e7bd-41cc-a079-841f77985e67 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:16:59.985252 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b40184-4b9d-480f-9c1f-07cec2242cd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:16:59.985891 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b40184-4b9d-480f-9c1f-07cec2242cd9 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:16:59.986174 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b40184-4b9d-480f-9c1f-07cec2242cd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:16:59.986451 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b40184-4b9d-480f-9c1f-07cec2242cd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:16:59.996388 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b40184-4b9d-480f-9c1f-07cec2242cd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:16:59.997827 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b40184-4b9d-480f-9c1f-07cec2242cd9 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:17:00.007569 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1437/2855] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:16:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:17:00.270529 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcbe628-296b-471e-b2d2-f372855be32f None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:17:00.272005 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-1bcbe628-296b-471e-b2d2-f372855be32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:17:00.294710 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcbe628-296b-471e-b2d2-f372855be32f None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:17:00.299342 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b819df5a-d4a9-4ebd-a75a-15738bb6e5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:17:00.300224 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b819df5a-d4a9-4ebd-a75a-15738bb6e5e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:17:00.300509 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b819df5a-d4a9-4ebd-a75a-15738bb6e5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:17:00.300786 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b819df5a-d4a9-4ebd-a75a-15738bb6e5e0 admin admin] PATH_INFO: `/v3/projects/25954e1508c340f1b78c3753ae9ac2db` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:17:00.301966 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b819df5a-d4a9-4ebd-a75a-15738bb6e5e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25954e1508c340f1b78c3753ae9ac2db)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:17:00.304699 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b819df5a-d4a9-4ebd-a75a-15738bb6e5e0 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:17:00.354415 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-b819df5a-d4a9-4ebd-a75a-15738bb6e5e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25954e1508c340f1b78c3753ae9ac2db', 'request_id': 'req-b819df5a-d4a9-4ebd-a75a-15738bb6e5e0'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:17:00.361239 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-b819df5a-d4a9-4ebd-a75a-15738bb6e5e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25954e1508c340f1b78c3753ae9ac2db', 'request_id': 'req-b819df5a-d4a9-4ebd-a75a-15738bb6e5e0'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:17:00.367987 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-b819df5a-d4a9-4ebd-a75a-15738bb6e5e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25954e1508c340f1b78c3753ae9ac2db', 'request_id': 'req-b819df5a-d4a9-4ebd-a75a-15738bb6e5e0'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:17:00.378528 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-b819df5a-d4a9-4ebd-a75a-15738bb6e5e0 admin admin] The token cache is being invalidate because project 25954e1508c340f1b78c3753ae9ac2db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:17:00.378822 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-b819df5a-d4a9-4ebd-a75a-15738bb6e5e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:17:00.379488 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-b819df5a-d4a9-4ebd-a75a-15738bb6e5e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:17:00.381051 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1419/2856] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:17:00 2022] DELETE /identity/v3/projects/25954e1508c340f1b78c3753ae9ac2db => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:17:00.385114 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b40184-4b9d-480f-9c1f-07cec2242cd9 service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:17:00.394609 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b40184-4b9d-480f-9c1f-07cec2242cd9 service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:17:00.449115 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c61f950-7025-4ef9-93cd-9d006442f01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:17:00.449974 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c61f950-7025-4ef9-93cd-9d006442f01c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:17:00.450256 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c61f950-7025-4ef9-93cd-9d006442f01c admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:17:00.450530 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c61f950-7025-4ef9-93cd-9d006442f01c admin admin] PATH_INFO: `/v3/users/972c5991960645de80141535724998be` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:17:00.451733 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c61f950-7025-4ef9-93cd-9d006442f01c admin admin] RBAC: Authorizing `identity:delete_user(user_id=972c5991960645de80141535724998be)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:17:00.453636 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c61f950-7025-4ef9-93cd-9d006442f01c admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:17:00.501278 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-5c61f950-7025-4ef9-93cd-9d006442f01c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '972c5991960645de80141535724998be'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:17:00.508509 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-5c61f950-7025-4ef9-93cd-9d006442f01c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '972c5991960645de80141535724998be'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:17:00.520069 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-5c61f950-7025-4ef9-93cd-9d006442f01c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '972c5991960645de80141535724998be'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:17:00.520750 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-5c61f950-7025-4ef9-93cd-9d006442f01c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '972c5991960645de80141535724998be'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:17:00.521346 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-5c61f950-7025-4ef9-93cd-9d006442f01c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '972c5991960645de80141535724998be'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:17:00.534531 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1438/2857] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:17:00 2022] DELETE /identity/v3/users/972c5991960645de80141535724998be => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:17:00.805642 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b819df5a-d4a9-4ebd-a75a-15738bb6e5e0 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:17:00.807247 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-b819df5a-d4a9-4ebd-a75a-15738bb6e5e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:17:00.824346 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b819df5a-d4a9-4ebd-a75a-15738bb6e5e0 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:17:00.888830 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a8e61f-823a-4487-8c89-8567ca4b3564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:17:00.889622 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a8e61f-823a-4487-8c89-8567ca4b3564 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:17:00.889903 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a8e61f-823a-4487-8c89-8567ca4b3564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:17:00.890178 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a8e61f-823a-4487-8c89-8567ca4b3564 admin admin] PATH_INFO: `/v3/projects/fe784a6f03d34bd1a6f7e7b933ab5c89` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:17:00.891382 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a8e61f-823a-4487-8c89-8567ca4b3564 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe784a6f03d34bd1a6f7e7b933ab5c89)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:17:00.893281 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a8e61f-823a-4487-8c89-8567ca4b3564 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:17:00.940148 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-57a8e61f-823a-4487-8c89-8567ca4b3564 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe784a6f03d34bd1a6f7e7b933ab5c89', 'request_id': 'req-57a8e61f-823a-4487-8c89-8567ca4b3564'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:17:00.947241 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-57a8e61f-823a-4487-8c89-8567ca4b3564 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe784a6f03d34bd1a6f7e7b933ab5c89', 'request_id': 'req-57a8e61f-823a-4487-8c89-8567ca4b3564'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:17:00.953451 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-57a8e61f-823a-4487-8c89-8567ca4b3564 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe784a6f03d34bd1a6f7e7b933ab5c89', 'request_id': 'req-57a8e61f-823a-4487-8c89-8567ca4b3564'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:17:00.959821 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:17:00.965570 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-57a8e61f-823a-4487-8c89-8567ca4b3564 admin admin] The token cache is being invalidate because project fe784a6f03d34bd1a6f7e7b933ab5c89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:17:00.965918 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-57a8e61f-823a-4487-8c89-8567ca4b3564 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:17:00.966980 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-57a8e61f-823a-4487-8c89-8567ca4b3564 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:17:00.968725 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1420/2858] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:17:00 2022] DELETE /identity/v3/projects/fe784a6f03d34bd1a6f7e7b933ab5c89 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:17:00.973635 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a467e450-48bb-4cde-b8bb-3308574b74b7 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:17:00.974165 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a467e450-48bb-4cde-b8bb-3308574b74b7 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:17:00.974165 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a467e450-48bb-4cde-b8bb-3308574b74b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:17:00.993428 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-a467e450-48bb-4cde-b8bb-3308574b74b7 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:17:00.994643 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a467e450-48bb-4cde-b8bb-3308574b74b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:17:01.022729 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:17:01.054838 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1439/2859] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:17:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:17:01.115053 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a1c95-0dee-461f-8116-f8119175a735 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:17:01.115346 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a1c95-0dee-461f-8116-f8119175a735 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:17:01.115650 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a1c95-0dee-461f-8116-f8119175a735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:17:01.134314 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-5f9a1c95-0dee-461f-8116-f8119175a735 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:17:01.135530 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-5f9a1c95-0dee-461f-8116-f8119175a735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:17:01.143709 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1421/2860] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:17:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:17:01.147905 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a467e450-48bb-4cde-b8bb-3308574b74b7 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:17:01.157449 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a467e450-48bb-4cde-b8bb-3308574b74b7 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:17:01.161996 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53a59d0-28bc-4713-89bf-fbb02a2a4926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:17:01.162639 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a59d0-28bc-4713-89bf-fbb02a2a4926 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:17:01.162931 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a59d0-28bc-4713-89bf-fbb02a2a4926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:17:01.163741 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a59d0-28bc-4713-89bf-fbb02a2a4926 admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:17:01.164464 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53a59d0-28bc-4713-89bf-fbb02a2a4926 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:17:01.164770 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53a59d0-28bc-4713-89bf-fbb02a2a4926 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:17:01.166354 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53a59d0-28bc-4713-89bf-fbb02a2a4926 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:17:01.177620 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1440/2861] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:17:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:17:01.182738 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a1c95-0dee-461f-8116-f8119175a735 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:17:01.191816 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a1c95-0dee-461f-8116-f8119175a735 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:17:01.197017 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda03178-69aa-4c99-abd9-b1dd2ca2b845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:17:01.197654 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda03178-69aa-4c99-abd9-b1dd2ca2b845 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:17:01.197936 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda03178-69aa-4c99-abd9-b1dd2ca2b845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:17:01.198212 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda03178-69aa-4c99-abd9-b1dd2ca2b845 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:17:01.198890 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda03178-69aa-4c99-abd9-b1dd2ca2b845 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:17:01.200794 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda03178-69aa-4c99-abd9-b1dd2ca2b845 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:17:01.226182 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1422/2862] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:17:01 2022] POST /identity/v3/projects => generated 355 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:17:01.231012 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53a59d0-28bc-4713-89bf-fbb02a2a4926 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:17:01.240829 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53a59d0-28bc-4713-89bf-fbb02a2a4926 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:17:01.297961 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3bfd0a-2c2a-4226-879a-0f53d0dee77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:17:01.298623 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bfd0a-2c2a-4226-879a-0f53d0dee77f admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:17:01.298907 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bfd0a-2c2a-4226-879a-0f53d0dee77f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:17:01.299179 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bfd0a-2c2a-4226-879a-0f53d0dee77f admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:17:01.299900 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3bfd0a-2c2a-4226-879a-0f53d0dee77f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:17:01.301797 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3bfd0a-2c2a-4226-879a-0f53d0dee77f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:17:01.321067 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1441/2863] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:17:01 2022] POST /identity/v3/users => generated 278 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:17:01.325108 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda03178-69aa-4c99-abd9-b1dd2ca2b845 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:17:01.335940 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda03178-69aa-4c99-abd9-b1dd2ca2b845 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:17:01.340576 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed857bf7-35af-4b2b-9eb3-ccf53d9a23f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:17:01.341231 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed857bf7-35af-4b2b-9eb3-ccf53d9a23f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:17:01.341516 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed857bf7-35af-4b2b-9eb3-ccf53d9a23f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:17:01.341789 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed857bf7-35af-4b2b-9eb3-ccf53d9a23f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:17:01.342542 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed857bf7-35af-4b2b-9eb3-ccf53d9a23f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:17:01.344141 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed857bf7-35af-4b2b-9eb3-ccf53d9a23f4 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:17:01.355261 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1423/2864] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:17:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:17:01.359023 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3bfd0a-2c2a-4226-879a-0f53d0dee77f admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:17:01.369821 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3bfd0a-2c2a-4226-879a-0f53d0dee77f admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:17:01.374859 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91c51d5-1374-485a-bf18-e7c372a88320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:17:01.375554 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c51d5-1374-485a-bf18-e7c372a88320 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:17:01.375840 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c51d5-1374-485a-bf18-e7c372a88320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:17:01.376117 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c51d5-1374-485a-bf18-e7c372a88320 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:17:01.376852 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91c51d5-1374-485a-bf18-e7c372a88320 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:17:01.378435 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91c51d5-1374-485a-bf18-e7c372a88320 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:17:01.389260 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1442/2865] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:17:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:17:01.393552 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed857bf7-35af-4b2b-9eb3-ccf53d9a23f4 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:17:01.404580 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed857bf7-35af-4b2b-9eb3-ccf53d9a23f4 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:17:01.409054 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f8311-b857-49d2-945a-e024146b81bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:17:01.409918 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f8311-b857-49d2-945a-e024146b81bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:17:01.410198 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f8311-b857-49d2-945a-e024146b81bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:17:01.410469 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f8311-b857-49d2-945a-e024146b81bc admin admin] PATH_INFO: `/v3/projects/097fd38932124a5f8a787e3d81994568/users/eff40863c75d46c1825766299623462b/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:17:01.425631 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296f8311-b857-49d2-945a-e024146b81bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=097fd38932124a5f8a787e3d81994568, user_id=eff40863c75d46c1825766299623462b, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:17:01.428303 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296f8311-b857-49d2-945a-e024146b81bc admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:17:01.440771 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1424/2866] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:17:01 2022] PUT /identity/v3/projects/097fd38932124a5f8a787e3d81994568/users/eff40863c75d46c1825766299623462b/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:17:01.450897 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed847937-5c00-4f81-8f46-24f38b2e0ff8 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:17:01.451184 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed847937-5c00-4f81-8f46-24f38b2e0ff8 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:17:01.451510 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed847937-5c00-4f81-8f46-24f38b2e0ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:17:01.477977 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-ed847937-5c00-4f81-8f46-24f38b2e0ff8 None None] MFA Rules not processed for user `eff40863c75d46c1825766299623462b`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:17:01.479465 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-ed847937-5c00-4f81-8f46-24f38b2e0ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:17:01.543493 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1443/2867] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:17:01 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:17:01.548064 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f8311-b857-49d2-945a-e024146b81bc admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:17:01.610994 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f8311-b857-49d2-945a-e024146b81bc admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:17:01.616503 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e56248-b6ec-4864-bd23-5e671e1c907c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:17:01.617345 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e56248-b6ec-4864-bd23-5e671e1c907c admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:17:01.617666 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e56248-b6ec-4864-bd23-5e671e1c907c admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:17:01.618205 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e56248-b6ec-4864-bd23-5e671e1c907c admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:17:01.619024 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e56248-b6ec-4864-bd23-5e671e1c907c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:17:01.621003 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e56248-b6ec-4864-bd23-5e671e1c907c admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:17:01.646361 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1425/2868] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:17:01 2022] POST /identity/v3/projects => generated 357 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:17:01.651143 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed847937-5c00-4f81-8f46-24f38b2e0ff8 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:17:01.661779 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed847937-5c00-4f81-8f46-24f38b2e0ff8 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:17:01.687540 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:17:01.719530 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cf40be-9d2d-45cd-a728-58d0c22c99fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:17:01.720188 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf40be-9d2d-45cd-a728-58d0c22c99fe admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:17:01.720468 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf40be-9d2d-45cd-a728-58d0c22c99fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:17:01.720740 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf40be-9d2d-45cd-a728-58d0c22c99fe admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:17:01.721414 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cf40be-9d2d-45cd-a728-58d0c22c99fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:17:01.723314 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cf40be-9d2d-45cd-a728-58d0c22c99fe admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:17:01.743992 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1444/2869] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:17:01 2022] POST /identity/v3/users => generated 285 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:17:01.747983 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e56248-b6ec-4864-bd23-5e671e1c907c admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:17:01.758707 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e56248-b6ec-4864-bd23-5e671e1c907c admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:17:01.763443 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce9fc96-d377-43bb-b307-33bed0516947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:17:01.764117 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce9fc96-d377-43bb-b307-33bed0516947 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:17:01.764404 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce9fc96-d377-43bb-b307-33bed0516947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:17:01.764685 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce9fc96-d377-43bb-b307-33bed0516947 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:17:01.765450 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce9fc96-d377-43bb-b307-33bed0516947 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:17:01.767058 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce9fc96-d377-43bb-b307-33bed0516947 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:17:01.779536 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1426/2870] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:17:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:17:01.783580 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cf40be-9d2d-45cd-a728-58d0c22c99fe admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:17:01.793596 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cf40be-9d2d-45cd-a728-58d0c22c99fe admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:17:01.798917 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba934bf-c0b0-47b2-9911-bf3882628075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:17:01.799832 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba934bf-c0b0-47b2-9911-bf3882628075 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:17:01.800102 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba934bf-c0b0-47b2-9911-bf3882628075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:17:01.800376 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba934bf-c0b0-47b2-9911-bf3882628075 admin admin] PATH_INFO: `/v3/projects/cb465d046f13459c8ca1e4809e780400/users/a7a42005ae26485eb300cbd406288564/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:17:01.816389 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba934bf-c0b0-47b2-9911-bf3882628075 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb465d046f13459c8ca1e4809e780400, user_id=a7a42005ae26485eb300cbd406288564, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:17:01.819070 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba934bf-c0b0-47b2-9911-bf3882628075 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:17:01.832505 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1445/2871] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:17:01 2022] PUT /identity/v3/projects/cb465d046f13459c8ca1e4809e780400/users/a7a42005ae26485eb300cbd406288564/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:17:01.837199 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce9fc96-d377-43bb-b307-33bed0516947 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:17:01.846937 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce9fc96-d377-43bb-b307-33bed0516947 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:17:01.903569 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455cacc0-2948-4aea-b7e8-618a9fb029dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:17:01.904453 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455cacc0-2948-4aea-b7e8-618a9fb029dc admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:17:01.904810 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455cacc0-2948-4aea-b7e8-618a9fb029dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:17:01.905115 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455cacc0-2948-4aea-b7e8-618a9fb029dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:17:01.905844 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455cacc0-2948-4aea-b7e8-618a9fb029dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:17:01.907467 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455cacc0-2948-4aea-b7e8-618a9fb029dc admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:17:01.917459 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1427/2872] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:17:01 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:17:01.921557 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba934bf-c0b0-47b2-9911-bf3882628075 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:17:01.931872 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba934bf-c0b0-47b2-9911-bf3882628075 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:17:01.936428 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd360676-88ef-4dd5-aa94-6c43f77f8d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:17:01.937291 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd360676-88ef-4dd5-aa94-6c43f77f8d47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:17:01.937566 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd360676-88ef-4dd5-aa94-6c43f77f8d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:17:01.937834 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd360676-88ef-4dd5-aa94-6c43f77f8d47 admin admin] PATH_INFO: `/v3/projects/cb465d046f13459c8ca1e4809e780400/users/a7a42005ae26485eb300cbd406288564/roles/6b483758dbd44eb4a78df2ce4603d601` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:17:01.939802 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd360676-88ef-4dd5-aa94-6c43f77f8d47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb465d046f13459c8ca1e4809e780400, user_id=a7a42005ae26485eb300cbd406288564, role_id=6b483758dbd44eb4a78df2ce4603d601)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:17:01.942391 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd360676-88ef-4dd5-aa94-6c43f77f8d47 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:17:01.953019 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1446/2873] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:17:01 2022] PUT /identity/v3/projects/cb465d046f13459c8ca1e4809e780400/users/a7a42005ae26485eb300cbd406288564/roles/6b483758dbd44eb4a78df2ce4603d601 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:17:01.963746 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ac46d-7887-4637-bd3b-b817f0ede4ea None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:17:01.964039 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ac46d-7887-4637-bd3b-b817f0ede4ea None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:17:01.964322 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ac46d-7887-4637-bd3b-b817f0ede4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:17:01.971032 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:17:01.992627 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-589ac46d-7887-4637-bd3b-b817f0ede4ea None None] MFA Rules not processed for user `a7a42005ae26485eb300cbd406288564`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:17:01.993841 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-589ac46d-7887-4637-bd3b-b817f0ede4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:17:02.059699 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1428/2874] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:17:01 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:17:02.071584 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd360676-88ef-4dd5-aa94-6c43f77f8d47 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:17:02.073415 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-cd360676-88ef-4dd5-aa94-6c43f77f8d47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:17:02.130268 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd360676-88ef-4dd5-aa94-6c43f77f8d47 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:17:02.134599 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f18a26-0432-4023-9155-98c73ab92c65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:17:02.135240 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f18a26-0432-4023-9155-98c73ab92c65 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:17:02.135559 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f18a26-0432-4023-9155-98c73ab92c65 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:17:02.135849 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f18a26-0432-4023-9155-98c73ab92c65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:17:02.144800 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f18a26-0432-4023-9155-98c73ab92c65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:17:02.146225 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f18a26-0432-4023-9155-98c73ab92c65 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:17:02.155566 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1447/2875] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:17:02 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:17:02.228670 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ac46d-7887-4637-bd3b-b817f0ede4ea None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:17:02.238431 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ac46d-7887-4637-bd3b-b817f0ede4ea None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:17:02.243025 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f859971a-e843-4f07-9b05-c462e3588c78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:17:02.243690 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859971a-e843-4f07-9b05-c462e3588c78 service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:17:02.243971 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859971a-e843-4f07-9b05-c462e3588c78 service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:17:02.244246 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859971a-e843-4f07-9b05-c462e3588c78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:17:02.300194 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f859971a-e843-4f07-9b05-c462e3588c78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:17:02.301653 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f859971a-e843-4f07-9b05-c462e3588c78 service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:17:02.311730 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1429/2876] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:17:02 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:17:12.151369 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=57,keystone:INSERT=5 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:17:12.307690 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=60 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:17:15.483890 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f18a26-0432-4023-9155-98c73ab92c65 service cinder] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:17:15.485288 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-65f18a26-0432-4023-9155-98c73ab92c65 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:17:15.578151 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f18a26-0432-4023-9155-98c73ab92c65 service cinder] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:17:15.583977 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ea8d8e-e838-4495-a001-5a2e940af9c2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02941c6ad2d54e48b1114d9e030c66c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:17:15.584632 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea8d8e-e838-4495-a001-5a2e940af9c2 service placement] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:17:15.584920 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea8d8e-e838-4495-a001-5a2e940af9c2 service placement] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:17:15.585202 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea8d8e-e838-4495-a001-5a2e940af9c2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:17:15.587035 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-e0ea8d8e-e838-4495-a001-5a2e940af9c2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:17:15.598906 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ea8d8e-e838-4495-a001-5a2e940af9c2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:17:15.600823 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ea8d8e-e838-4495-a001-5a2e940af9c2 service placement] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:17:15.611785 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1448/2877] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:17:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:17:17.191071 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f859971a-e843-4f07-9b05-c462e3588c78 service cinder] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:17:17.254070 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f859971a-e843-4f07-9b05-c462e3588c78 service cinder] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:17:17.258710 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6203612-75cc-44b2-a5c8-0fddd9e7da65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:17:17.259599 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6203612-75cc-44b2-a5c8-0fddd9e7da65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:17:17.259883 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6203612-75cc-44b2-a5c8-0fddd9e7da65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:17:17.260159 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6203612-75cc-44b2-a5c8-0fddd9e7da65 admin admin] PATH_INFO: `/v3/users/eff40863c75d46c1825766299623462b` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:17:17.261327 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6203612-75cc-44b2-a5c8-0fddd9e7da65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eff40863c75d46c1825766299623462b)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:17:17.263265 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6203612-75cc-44b2-a5c8-0fddd9e7da65 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:17:17.308807 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-b6203612-75cc-44b2-a5c8-0fddd9e7da65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eff40863c75d46c1825766299623462b'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:17:17.309450 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-b6203612-75cc-44b2-a5c8-0fddd9e7da65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eff40863c75d46c1825766299623462b'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:17:17.316773 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-b6203612-75cc-44b2-a5c8-0fddd9e7da65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eff40863c75d46c1825766299623462b'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:17:17.328541 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-b6203612-75cc-44b2-a5c8-0fddd9e7da65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eff40863c75d46c1825766299623462b'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:17:17.336248 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-b6203612-75cc-44b2-a5c8-0fddd9e7da65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eff40863c75d46c1825766299623462b'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:17:17.342279 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1430/2878] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:17:17 2022] DELETE /identity/v3/users/eff40863c75d46c1825766299623462b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:17:17.346684 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2063e6de-39bc-4839-ab4e-c488c670239a None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:17:17.347168 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2063e6de-39bc-4839-ab4e-c488c670239a None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:17:17.347283 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2063e6de-39bc-4839-ab4e-c488c670239a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:17:17.365150 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-2063e6de-39bc-4839-ab4e-c488c670239a None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:17:17.366383 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-2063e6de-39bc-4839-ab4e-c488c670239a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:17:17.426022 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1449/2879] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:17:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:17:17.435696 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6203612-75cc-44b2-a5c8-0fddd9e7da65 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:17:17.437087 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-b6203612-75cc-44b2-a5c8-0fddd9e7da65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:17:17.492835 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6203612-75cc-44b2-a5c8-0fddd9e7da65 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:17:17.496437 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c614de-4d54-4df9-9bfa-4042a9bc1fd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:17:17.497072 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c614de-4d54-4df9-9bfa-4042a9bc1fd9 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:17:17.497355 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c614de-4d54-4df9-9bfa-4042a9bc1fd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:17:17.497630 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c614de-4d54-4df9-9bfa-4042a9bc1fd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:17:17.506552 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c614de-4d54-4df9-9bfa-4042a9bc1fd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:17:17.508595 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c614de-4d54-4df9-9bfa-4042a9bc1fd9 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:17:17.518034 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1431/2880] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:17:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:17:17.781986 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2063e6de-39bc-4839-ab4e-c488c670239a None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:17:17.783483 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-2063e6de-39bc-4839-ab4e-c488c670239a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:17:17.795604 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2063e6de-39bc-4839-ab4e-c488c670239a None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:17:17.800203 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e02cc7-9039-430c-8ccc-e67f54d3b39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:17:17.801062 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e02cc7-9039-430c-8ccc-e67f54d3b39a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:17:17.801342 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e02cc7-9039-430c-8ccc-e67f54d3b39a admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:17:17.801618 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e02cc7-9039-430c-8ccc-e67f54d3b39a admin admin] PATH_INFO: `/v3/projects/097fd38932124a5f8a787e3d81994568` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:17:17.810212 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e02cc7-9039-430c-8ccc-e67f54d3b39a admin admin] RBAC: Authorizing `identity:delete_project(project_id=097fd38932124a5f8a787e3d81994568)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:17:17.812249 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e02cc7-9039-430c-8ccc-e67f54d3b39a admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:17:17.858673 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-99e02cc7-9039-430c-8ccc-e67f54d3b39a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '097fd38932124a5f8a787e3d81994568', 'request_id': 'req-99e02cc7-9039-430c-8ccc-e67f54d3b39a'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:17:17.866107 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-99e02cc7-9039-430c-8ccc-e67f54d3b39a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '097fd38932124a5f8a787e3d81994568', 'request_id': 'req-99e02cc7-9039-430c-8ccc-e67f54d3b39a'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:17:17.872318 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-99e02cc7-9039-430c-8ccc-e67f54d3b39a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '097fd38932124a5f8a787e3d81994568', 'request_id': 'req-99e02cc7-9039-430c-8ccc-e67f54d3b39a'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:17:17.873418 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:17:17.886818 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-99e02cc7-9039-430c-8ccc-e67f54d3b39a admin admin] The token cache is being invalidate because project 097fd38932124a5f8a787e3d81994568 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:17:17.887108 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-99e02cc7-9039-430c-8ccc-e67f54d3b39a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:17:17.887770 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-99e02cc7-9039-430c-8ccc-e67f54d3b39a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:17:17.889288 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1450/2881] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:17:17 2022] DELETE /identity/v3/projects/097fd38932124a5f8a787e3d81994568 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:17:17.893228 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c614de-4d54-4df9-9bfa-4042a9bc1fd9 service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:17:17.902975 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c614de-4d54-4df9-9bfa-4042a9bc1fd9 service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:17:17.922525 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:17:17.967059 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7cfd99-69dc-4db4-bd8b-ee0321ea4e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:17:17.967960 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7cfd99-69dc-4db4-bd8b-ee0321ea4e3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:17:17.968243 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7cfd99-69dc-4db4-bd8b-ee0321ea4e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:17:17.968519 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7cfd99-69dc-4db4-bd8b-ee0321ea4e3b admin admin] PATH_INFO: `/v3/users/a7a42005ae26485eb300cbd406288564` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:17:17.969703 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7cfd99-69dc-4db4-bd8b-ee0321ea4e3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7a42005ae26485eb300cbd406288564)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:17:17.971640 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7cfd99-69dc-4db4-bd8b-ee0321ea4e3b admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:17:18.017074 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-1c7cfd99-69dc-4db4-bd8b-ee0321ea4e3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7a42005ae26485eb300cbd406288564'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:17:18.017708 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-1c7cfd99-69dc-4db4-bd8b-ee0321ea4e3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7a42005ae26485eb300cbd406288564'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:17:18.024867 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-1c7cfd99-69dc-4db4-bd8b-ee0321ea4e3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7a42005ae26485eb300cbd406288564'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:17:18.036578 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-1c7cfd99-69dc-4db4-bd8b-ee0321ea4e3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7a42005ae26485eb300cbd406288564'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:17:18.043479 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-1c7cfd99-69dc-4db4-bd8b-ee0321ea4e3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7a42005ae26485eb300cbd406288564'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:17:18.049249 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1432/2882] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:17:17 2022] DELETE /identity/v3/users/a7a42005ae26485eb300cbd406288564 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:17:18.330853 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e02cc7-9039-430c-8ccc-e67f54d3b39a admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:17:18.332959 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-99e02cc7-9039-430c-8ccc-e67f54d3b39a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:17:18.349814 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e02cc7-9039-430c-8ccc-e67f54d3b39a admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:17:18.415482 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57eb845-3eeb-4e11-a16e-6a467895919a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:17:18.416347 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57eb845-3eeb-4e11-a16e-6a467895919a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:17:18.416632 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57eb845-3eeb-4e11-a16e-6a467895919a admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:17:18.416908 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57eb845-3eeb-4e11-a16e-6a467895919a admin admin] PATH_INFO: `/v3/projects/cb465d046f13459c8ca1e4809e780400` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:17:18.418097 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57eb845-3eeb-4e11-a16e-6a467895919a admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb465d046f13459c8ca1e4809e780400)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:17:18.420033 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57eb845-3eeb-4e11-a16e-6a467895919a admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:17:18.465813 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-e57eb845-3eeb-4e11-a16e-6a467895919a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb465d046f13459c8ca1e4809e780400', 'request_id': 'req-e57eb845-3eeb-4e11-a16e-6a467895919a'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:17:18.472547 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-e57eb845-3eeb-4e11-a16e-6a467895919a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb465d046f13459c8ca1e4809e780400', 'request_id': 'req-e57eb845-3eeb-4e11-a16e-6a467895919a'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:17:18.478860 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-e57eb845-3eeb-4e11-a16e-6a467895919a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb465d046f13459c8ca1e4809e780400', 'request_id': 'req-e57eb845-3eeb-4e11-a16e-6a467895919a'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:17:18.489887 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-e57eb845-3eeb-4e11-a16e-6a467895919a admin admin] The token cache is being invalidate because project cb465d046f13459c8ca1e4809e780400 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:17:18.490174 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-e57eb845-3eeb-4e11-a16e-6a467895919a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:17:18.490795 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-e57eb845-3eeb-4e11-a16e-6a467895919a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:17:18.492802 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1451/2883] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:17:18 2022] DELETE /identity/v3/projects/cb465d046f13459c8ca1e4809e780400 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:17:18.497742 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb0740-2914-45d0-8f30-7fa3d06f2e15 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:17:18.498232 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb0740-2914-45d0-8f30-7fa3d06f2e15 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:17:18.498338 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb0740-2914-45d0-8f30-7fa3d06f2e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:17:18.516210 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-d4bb0740-2914-45d0-8f30-7fa3d06f2e15 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:17:18.517397 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-d4bb0740-2914-45d0-8f30-7fa3d06f2e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:17:18.574796 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1433/2884] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:17:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:17:18.582654 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c055350-cd09-4da9-94fe-55acd43d4629 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:17:18.582948 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c055350-cd09-4da9-94fe-55acd43d4629 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:17:18.583224 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c055350-cd09-4da9-94fe-55acd43d4629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:17:18.602496 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-5c055350-cd09-4da9-94fe-55acd43d4629 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:17:18.603725 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-5c055350-cd09-4da9-94fe-55acd43d4629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:17:18.612017 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1452/2885] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:17:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:17:18.616242 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bb0740-2914-45d0-8f30-7fa3d06f2e15 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:17:18.627110 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bb0740-2914-45d0-8f30-7fa3d06f2e15 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:17:18.631790 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6509a3a2-cca1-410d-92db-b41d23a8dcd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:17:18.632463 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6509a3a2-cca1-410d-92db-b41d23a8dcd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:17:18.632747 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6509a3a2-cca1-410d-92db-b41d23a8dcd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:17:18.633028 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6509a3a2-cca1-410d-92db-b41d23a8dcd3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:17:18.633768 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6509a3a2-cca1-410d-92db-b41d23a8dcd3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 22:17:18.634074 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6509a3a2-cca1-410d-92db-b41d23a8dcd3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:17:18.635729 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6509a3a2-cca1-410d-92db-b41d23a8dcd3 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:17:18.647661 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1434/2886] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jun 7 22:17:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 22:17:18.652573 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c055350-cd09-4da9-94fe-55acd43d4629 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:17:18.661795 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c055350-cd09-4da9-94fe-55acd43d4629 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:17:18.667126 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be37487-389e-4caa-9849-cf86efa412da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:17:18.667802 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be37487-389e-4caa-9849-cf86efa412da admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:17:18.668088 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be37487-389e-4caa-9849-cf86efa412da admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:17:18.668363 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be37487-389e-4caa-9849-cf86efa412da admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:17:18.669083 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be37487-389e-4caa-9849-cf86efa412da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:17:18.670970 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be37487-389e-4caa-9849-cf86efa412da admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:17:18.695873 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1453/2887] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jun 7 22:17:18 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:17:18.700041 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6509a3a2-cca1-410d-92db-b41d23a8dcd3 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:17:18.709834 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6509a3a2-cca1-410d-92db-b41d23a8dcd3 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:17:18.738818 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:17:18.771144 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f9517e-fb03-411d-910e-4da604ac8bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:17:18.771857 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9517e-fb03-411d-910e-4da604ac8bb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:17:18.772142 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9517e-fb03-411d-910e-4da604ac8bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:17:18.772419 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9517e-fb03-411d-910e-4da604ac8bb6 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:17:18.773651 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f9517e-fb03-411d-910e-4da604ac8bb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:17:18.775768 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f9517e-fb03-411d-910e-4da604ac8bb6 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:17:18.798571 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1435/2888] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jun 7 22:17:18 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 22:17:18.802675 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be37487-389e-4caa-9849-cf86efa412da admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:17:18.813574 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be37487-389e-4caa-9849-cf86efa412da admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:17:18.818238 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64acd990-ff97-4281-bc24-c7581338e5b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:17:18.818889 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64acd990-ff97-4281-bc24-c7581338e5b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:17:18.819173 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64acd990-ff97-4281-bc24-c7581338e5b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:17:18.819492 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64acd990-ff97-4281-bc24-c7581338e5b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:17:18.820220 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64acd990-ff97-4281-bc24-c7581338e5b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:17:18.821811 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64acd990-ff97-4281-bc24-c7581338e5b2 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:17:18.832131 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1454/2889] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:17:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:17:18.835878 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f9517e-fb03-411d-910e-4da604ac8bb6 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:17:18.846528 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f9517e-fb03-411d-910e-4da604ac8bb6 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:17:18.851063 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d971eb-6ec3-48a7-8f64-88d263cd5f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:17:18.851739 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d971eb-6ec3-48a7-8f64-88d263cd5f28 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:17:18.852636 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d971eb-6ec3-48a7-8f64-88d263cd5f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:17:18.852907 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d971eb-6ec3-48a7-8f64-88d263cd5f28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:17:18.853621 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d971eb-6ec3-48a7-8f64-88d263cd5f28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:17:18.855472 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d971eb-6ec3-48a7-8f64-88d263cd5f28 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:17:18.865518 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1436/2890] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jun 7 22:17:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 22:17:18.869566 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64acd990-ff97-4281-bc24-c7581338e5b2 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:17:18.879531 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64acd990-ff97-4281-bc24-c7581338e5b2 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:17:18.884209 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de203242-54ed-4d80-8781-e8937dd4fabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:17:18.885073 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de203242-54ed-4d80-8781-e8937dd4fabe admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:17:18.885354 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de203242-54ed-4d80-8781-e8937dd4fabe admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:17:18.885628 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de203242-54ed-4d80-8781-e8937dd4fabe admin admin] PATH_INFO: `/v3/projects/4a09bf9290a741b381ba33f5622d6210/users/e929aea446f74cbb9e0247f651fdf162/roles/c5431c9deadf47e496264ba5b78eeaf6` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:17:18.900287 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de203242-54ed-4d80-8781-e8937dd4fabe admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a09bf9290a741b381ba33f5622d6210, user_id=e929aea446f74cbb9e0247f651fdf162, role_id=c5431c9deadf47e496264ba5b78eeaf6)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:17:18.902902 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de203242-54ed-4d80-8781-e8937dd4fabe admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:17:18.915998 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1455/2891] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jun 7 22:17:18 2022] PUT /identity/v3/projects/4a09bf9290a741b381ba33f5622d6210/users/e929aea446f74cbb9e0247f651fdf162/roles/c5431c9deadf47e496264ba5b78eeaf6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:17:18.926085 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce483f5a-5625-407c-b94e-8bbc4445ef24 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:17:18.926378 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce483f5a-5625-407c-b94e-8bbc4445ef24 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:17:18.926657 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce483f5a-5625-407c-b94e-8bbc4445ef24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:17:18.952136 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-ce483f5a-5625-407c-b94e-8bbc4445ef24 None None] MFA Rules not processed for user `e929aea446f74cbb9e0247f651fdf162`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:17:18.953323 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-ce483f5a-5625-407c-b94e-8bbc4445ef24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:17:19.018343 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1437/2892] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:17:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:17:19.029882 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de203242-54ed-4d80-8781-e8937dd4fabe admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:17:19.031693 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-de203242-54ed-4d80-8781-e8937dd4fabe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:17:19.053791 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:17:19.091688 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de203242-54ed-4d80-8781-e8937dd4fabe admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:17:19.095294 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28668304-6f54-4d8d-b25b-19ad92f9c5b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8a1e2cf1da4f87b8d4e16f525fc759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:17:19.095961 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28668304-6f54-4d8d-b25b-19ad92f9c5b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:17:19.096243 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28668304-6f54-4d8d-b25b-19ad92f9c5b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:17:19.096515 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28668304-6f54-4d8d-b25b-19ad92f9c5b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:17:19.105847 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28668304-6f54-4d8d-b25b-19ad92f9c5b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:17:19.107262 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28668304-6f54-4d8d-b25b-19ad92f9c5b5 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:17:19.116073 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1456/2893] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:17:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:17:27.456533 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce483f5a-5625-407c-b94e-8bbc4445ef24 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:17:27.518497 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce483f5a-5625-407c-b94e-8bbc4445ef24 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:17:27.523056 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f758689c-7c3e-452f-902c-f2fcb0ef2a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:17:27.524885 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758689c-7c3e-452f-902c-f2fcb0ef2a13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:17:27.525169 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758689c-7c3e-452f-902c-f2fcb0ef2a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:17:27.525446 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758689c-7c3e-452f-902c-f2fcb0ef2a13 admin admin] PATH_INFO: `/v3/users/e929aea446f74cbb9e0247f651fdf162` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:17:27.526731 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f758689c-7c3e-452f-902c-f2fcb0ef2a13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e929aea446f74cbb9e0247f651fdf162)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:17:27.528653 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f758689c-7c3e-452f-902c-f2fcb0ef2a13 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:17:27.568462 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:17:27.575743 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-f758689c-7c3e-452f-902c-f2fcb0ef2a13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e929aea446f74cbb9e0247f651fdf162'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:17:27.576684 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-f758689c-7c3e-452f-902c-f2fcb0ef2a13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e929aea446f74cbb9e0247f651fdf162'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:17:27.583492 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-f758689c-7c3e-452f-902c-f2fcb0ef2a13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e929aea446f74cbb9e0247f651fdf162'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:17:27.596080 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-f758689c-7c3e-452f-902c-f2fcb0ef2a13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e929aea446f74cbb9e0247f651fdf162'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:17:27.603903 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-f758689c-7c3e-452f-902c-f2fcb0ef2a13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e929aea446f74cbb9e0247f651fdf162'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:17:27.609970 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1438/2894] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:17:27 2022] DELETE /identity/v3/users/e929aea446f74cbb9e0247f651fdf162 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:17:27.614434 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a2aa21-89ee-454f-b3da-955220dfb768 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:17:27.614931 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a2aa21-89ee-454f-b3da-955220dfb768 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:17:27.615044 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a2aa21-89ee-454f-b3da-955220dfb768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:17:27.634670 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-46a2aa21-89ee-454f-b3da-955220dfb768 None None] MFA Rules not processed for user `c9d023b1a9bd4c969261ea4adeb6b556`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 22:17:27.635894 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-46a2aa21-89ee-454f-b3da-955220dfb768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:17:27.693521 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1457/2895] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jun 7 22:17:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 22:17:27.703190 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f758689c-7c3e-452f-902c-f2fcb0ef2a13 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:17:27.704612 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-f758689c-7c3e-452f-902c-f2fcb0ef2a13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:17:27.763818 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f758689c-7c3e-452f-902c-f2fcb0ef2a13 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:17:27.767404 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f06fe0d-0caa-4d03-882b-eef43e12db3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:17:27.768055 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f06fe0d-0caa-4d03-882b-eef43e12db3e service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:17:27.768333 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f06fe0d-0caa-4d03-882b-eef43e12db3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:17:27.768608 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f06fe0d-0caa-4d03-882b-eef43e12db3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:17:27.777832 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f06fe0d-0caa-4d03-882b-eef43e12db3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:17:27.779244 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f06fe0d-0caa-4d03-882b-eef43e12db3e service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:17:27.788493 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1439/2896] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:17:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:17:28.056932 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a2aa21-89ee-454f-b3da-955220dfb768 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:17:28.058383 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-46a2aa21-89ee-454f-b3da-955220dfb768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:17:28.070569 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a2aa21-89ee-454f-b3da-955220dfb768 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:17:28.075187 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639de0e-665c-40e6-99c9-542980789ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:17:28.076076 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639de0e-665c-40e6-99c9-542980789ecb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:17:28.076360 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639de0e-665c-40e6-99c9-542980789ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:17:28.076636 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639de0e-665c-40e6-99c9-542980789ecb admin admin] PATH_INFO: `/v3/projects/4a09bf9290a741b381ba33f5622d6210` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:17:28.077818 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3639de0e-665c-40e6-99c9-542980789ecb admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a09bf9290a741b381ba33f5622d6210)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:17:28.080481 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3639de0e-665c-40e6-99c9-542980789ecb admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:17:28.127258 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3639de0e-665c-40e6-99c9-542980789ecb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a09bf9290a741b381ba33f5622d6210', 'request_id': 'req-3639de0e-665c-40e6-99c9-542980789ecb'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:17:28.134469 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3639de0e-665c-40e6-99c9-542980789ecb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a09bf9290a741b381ba33f5622d6210', 'request_id': 'req-3639de0e-665c-40e6-99c9-542980789ecb'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:17:28.140015 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3639de0e-665c-40e6-99c9-542980789ecb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a09bf9290a741b381ba33f5622d6210', 'request_id': 'req-3639de0e-665c-40e6-99c9-542980789ecb'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:17:28.150796 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3639de0e-665c-40e6-99c9-542980789ecb admin admin] The token cache is being invalidate because project 4a09bf9290a741b381ba33f5622d6210 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:17:28.151081 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3639de0e-665c-40e6-99c9-542980789ecb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:17:28.151725 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3639de0e-665c-40e6-99c9-542980789ecb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:17:28.153235 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1458/2897] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:17:28 2022] DELETE /identity/v3/projects/4a09bf9290a741b381ba33f5622d6210 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 22:17:37.784391 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:DELETE=5,keystone:SELECT=40,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:17:38.147759 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:17:50.720622 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f06fe0d-0caa-4d03-882b-eef43e12db3e service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:17:50.722045 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-2f06fe0d-0caa-4d03-882b-eef43e12db3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:17:50.734479 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f06fe0d-0caa-4d03-882b-eef43e12db3e service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:17:50.782652 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d945db4f-669e-49ba-b77a-8799554a6454 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:17:50.783305 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d945db4f-669e-49ba-b77a-8799554a6454 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:17:50.783617 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d945db4f-669e-49ba-b77a-8799554a6454 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:17:50.783899 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d945db4f-669e-49ba-b77a-8799554a6454 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:17:50.785675 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-d945db4f-669e-49ba-b77a-8799554a6454 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:17:50.842552 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d945db4f-669e-49ba-b77a-8799554a6454 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:17:50.844009 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d945db4f-669e-49ba-b77a-8799554a6454 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:17:50.852786 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1440/2898] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:17:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:17:51.913304 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639de0e-665c-40e6-99c9-542980789ecb admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:17:51.915124 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-3639de0e-665c-40e6-99c9-542980789ecb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:17:52.046635 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639de0e-665c-40e6-99c9-542980789ecb admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:17:52.053068 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c265091-9f62-463a-9915-a636d189378f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe0303ea0764347bfb054870ebba41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:17:52.053787 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c265091-9f62-463a-9915-a636d189378f service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:17:52.054076 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c265091-9f62-463a-9915-a636d189378f service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:17:52.054354 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c265091-9f62-463a-9915-a636d189378f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:17:52.056232 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-8c265091-9f62-463a-9915-a636d189378f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:17:52.074092 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c265091-9f62-463a-9915-a636d189378f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:17:52.075761 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c265091-9f62-463a-9915-a636d189378f service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:17:52.091512 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1459/2899] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:17:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:17:55.781285 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d945db4f-669e-49ba-b77a-8799554a6454 service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:17:55.782730 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-d945db4f-669e-49ba-b77a-8799554a6454 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:17:55.794080 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d945db4f-669e-49ba-b77a-8799554a6454 service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:17:55.797721 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74491880-e8e1-49e3-8157-865c02c14525 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10242251e6284217bfb79c67b1567f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:17:55.798361 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74491880-e8e1-49e3-8157-865c02c14525 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:17:55.798643 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74491880-e8e1-49e3-8157-865c02c14525 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:17:55.798922 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74491880-e8e1-49e3-8157-865c02c14525 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:17:55.800729 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-74491880-e8e1-49e3-8157-865c02c14525 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:17:55.863344 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74491880-e8e1-49e3-8157-865c02c14525 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:17:55.864803 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74491880-e8e1-49e3-8157-865c02c14525 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:17:55.875139 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1441/2900] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:17:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:18:01.840586 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c265091-9f62-463a-9915-a636d189378f service nova] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:18:01.842207 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-8c265091-9f62-463a-9915-a636d189378f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:18:01.856888 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c265091-9f62-463a-9915-a636d189378f service nova] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:18:01.862409 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2267cd-1434-4b42-a93d-c98129677a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:18:01.863484 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2267cd-1434-4b42-a93d-c98129677a24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:18:01.863773 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2267cd-1434-4b42-a93d-c98129677a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:18:01.864055 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2267cd-1434-4b42-a93d-c98129677a24 admin admin] PATH_INFO: `/v3/users/7e8c46b1820a4ce5ac853ab1cc88fc30` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:18:01.865319 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2267cd-1434-4b42-a93d-c98129677a24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e8c46b1820a4ce5ac853ab1cc88fc30)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:18:01.867257 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2267cd-1434-4b42-a93d-c98129677a24 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:18:01.920437 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-6b2267cd-1434-4b42-a93d-c98129677a24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e8c46b1820a4ce5ac853ab1cc88fc30'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:18:01.927792 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-6b2267cd-1434-4b42-a93d-c98129677a24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e8c46b1820a4ce5ac853ab1cc88fc30'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:18:01.940758 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-6b2267cd-1434-4b42-a93d-c98129677a24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e8c46b1820a4ce5ac853ab1cc88fc30'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:18:01.941566 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-6b2267cd-1434-4b42-a93d-c98129677a24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e8c46b1820a4ce5ac853ab1cc88fc30'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:18:01.942275 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-6b2267cd-1434-4b42-a93d-c98129677a24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e8c46b1820a4ce5ac853ab1cc88fc30'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:18:01.956901 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1460/2901] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:18:01 2022] DELETE /identity/v3/users/7e8c46b1820a4ce5ac853ab1cc88fc30 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:18:02.130150 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74491880-e8e1-49e3-8157-865c02c14525 service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:18:02.143172 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74491880-e8e1-49e3-8157-865c02c14525 service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:18:02.189334 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:18:02.201602 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d96696d-663f-40b5-a838-7771c42dc04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:18:02.202454 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96696d-663f-40b5-a838-7771c42dc04c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:18:02.202734 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96696d-663f-40b5-a838-7771c42dc04c admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:18:02.203007 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96696d-663f-40b5-a838-7771c42dc04c admin admin] PATH_INFO: `/v3/projects/98914183afee4e58ad7a425cb019e639` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:18:02.204209 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d96696d-663f-40b5-a838-7771c42dc04c admin admin] RBAC: Authorizing `identity:delete_project(project_id=98914183afee4e58ad7a425cb019e639)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:18:02.206096 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d96696d-663f-40b5-a838-7771c42dc04c admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:18:02.251120 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-5d96696d-663f-40b5-a838-7771c42dc04c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98914183afee4e58ad7a425cb019e639', 'request_id': 'req-5d96696d-663f-40b5-a838-7771c42dc04c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:18:02.257983 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-5d96696d-663f-40b5-a838-7771c42dc04c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98914183afee4e58ad7a425cb019e639', 'request_id': 'req-5d96696d-663f-40b5-a838-7771c42dc04c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:18:02.264617 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-5d96696d-663f-40b5-a838-7771c42dc04c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98914183afee4e58ad7a425cb019e639', 'request_id': 'req-5d96696d-663f-40b5-a838-7771c42dc04c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:18:02.274453 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-5d96696d-663f-40b5-a838-7771c42dc04c admin admin] The token cache is being invalidate because project 98914183afee4e58ad7a425cb019e639 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:18:02.274736 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-5d96696d-663f-40b5-a838-7771c42dc04c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:18:02.275379 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-5d96696d-663f-40b5-a838-7771c42dc04c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:18:02.276876 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1442/2902] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:18:02 2022] DELETE /identity/v3/projects/98914183afee4e58ad7a425cb019e639 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:18:02.280956 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2267cd-1434-4b42-a93d-c98129677a24 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:18:02.282344 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-6b2267cd-1434-4b42-a93d-c98129677a24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:18:02.296352 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2267cd-1434-4b42-a93d-c98129677a24 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:18:02.325429 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:18:02.353605 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2582aa-aabc-47b7-afe8-b802828f68df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:18:02.354752 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2582aa-aabc-47b7-afe8-b802828f68df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:18:02.355036 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2582aa-aabc-47b7-afe8-b802828f68df admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:18:02.355901 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2582aa-aabc-47b7-afe8-b802828f68df admin admin] PATH_INFO: `/v3/users/df8302231ba9470c99b27bd0048e983b` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:18:02.357219 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2582aa-aabc-47b7-afe8-b802828f68df admin admin] RBAC: Authorizing `identity:delete_user(user_id=df8302231ba9470c99b27bd0048e983b)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:18:02.360058 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2582aa-aabc-47b7-afe8-b802828f68df admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:18:02.407058 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-bc2582aa-aabc-47b7-afe8-b802828f68df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df8302231ba9470c99b27bd0048e983b'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:18:02.413679 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-bc2582aa-aabc-47b7-afe8-b802828f68df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df8302231ba9470c99b27bd0048e983b'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:18:02.426355 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-bc2582aa-aabc-47b7-afe8-b802828f68df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df8302231ba9470c99b27bd0048e983b'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:18:02.427000 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-bc2582aa-aabc-47b7-afe8-b802828f68df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df8302231ba9470c99b27bd0048e983b'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:18:02.427610 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-bc2582aa-aabc-47b7-afe8-b802828f68df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df8302231ba9470c99b27bd0048e983b'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:18:02.440550 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1461/2903] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jun 7 22:18:02 2022] DELETE /identity/v3/users/df8302231ba9470c99b27bd0048e983b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:18:02.610662 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d96696d-663f-40b5-a838-7771c42dc04c admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:18:02.612092 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-5d96696d-663f-40b5-a838-7771c42dc04c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:18:02.626695 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d96696d-663f-40b5-a838-7771c42dc04c admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:18:02.684601 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1346459e-e458-4ea2-8a73-a76730a50bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9d023b1a9bd4c969261ea4adeb6b556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c61211fad6e447b0bf162f99f1111a82', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:18:02.685397 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1346459e-e458-4ea2-8a73-a76730a50bdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:18:02.685686 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1346459e-e458-4ea2-8a73-a76730a50bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:18:02.685968 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1346459e-e458-4ea2-8a73-a76730a50bdc admin admin] PATH_INFO: `/v3/projects/483ce8210232462f81b34fe3973a3cfd` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:18:02.687150 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1346459e-e458-4ea2-8a73-a76730a50bdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=483ce8210232462f81b34fe3973a3cfd)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:18:02.689107 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1346459e-e458-4ea2-8a73-a76730a50bdc admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:18:02.736319 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-1346459e-e458-4ea2-8a73-a76730a50bdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '483ce8210232462f81b34fe3973a3cfd', 'request_id': 'req-1346459e-e458-4ea2-8a73-a76730a50bdc'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:18:02.743129 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-1346459e-e458-4ea2-8a73-a76730a50bdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '483ce8210232462f81b34fe3973a3cfd', 'request_id': 'req-1346459e-e458-4ea2-8a73-a76730a50bdc'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:18:02.749708 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-1346459e-e458-4ea2-8a73-a76730a50bdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '483ce8210232462f81b34fe3973a3cfd', 'request_id': 'req-1346459e-e458-4ea2-8a73-a76730a50bdc'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:18:02.760757 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-1346459e-e458-4ea2-8a73-a76730a50bdc admin admin] The token cache is being invalidate because project 483ce8210232462f81b34fe3973a3cfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 22:18:02.761063 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-1346459e-e458-4ea2-8a73-a76730a50bdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:18:02.761765 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-1346459e-e458-4ea2-8a73-a76730a50bdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 22:18:02.763629 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1443/2904] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jun 7 22:18:02 2022] DELETE /identity/v3/projects/483ce8210232462f81b34fe3973a3cfd => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 22:18:12.436991 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=39,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:18:12.757384 dv-cinder-835525-4-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=80,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 22:22:19.465063 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2582aa-aabc-47b7-afe8-b802828f68df admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 22:22:19.466439 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-bc2582aa-aabc-47b7-afe8-b802828f68df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:22:19.536321 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2582aa-aabc-47b7-afe8-b802828f68df admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 22:22:19.595629 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e4853-fa80-4d42-aa92-d16a3591aa1b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02941c6ad2d54e48b1114d9e030c66c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0219dbda54274fdc9bdccbd960c30c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 22:22:19.596286 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e4853-fa80-4d42-aa92-d16a3591aa1b service placement] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 22:22:19.596568 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e4853-fa80-4d42-aa92-d16a3591aa1b service placement] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 22:22:19.596845 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e4853-fa80-4d42-aa92-d16a3591aa1b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 22:22:19.598634 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-525e4853-fa80-4d42-aa92-d16a3591aa1b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 22:22:19.609955 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525e4853-fa80-4d42-aa92-d16a3591aa1b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 22:22:19.611562 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525e4853-fa80-4d42-aa92-d16a3591aa1b service placement] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 22:22:19.621352 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1462/2905] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jun 7 22:22:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 22:22:29.617152 dv-cinder-835525-4-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=58 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}